Skip to content

Adopt the py-canon standard, and read Parquet - #17

Merged
soodoku merged 2 commits into
mainfrom
adopt-py-canon
Aug 20, 2026
Merged

Adopt the py-canon standard, and read Parquet#17
soodoku merged 2 commits into
mainfrom
adopt-py-canon

Conversation

@soodoku

@soodoku soodoku commented Aug 20, 2026

Copy link
Copy Markdown
Member

Three things that had to happen together, because the standard's linters are what surfaced the first one.

A bug: max_dependence_points did nothing

estimate(max_dependence_points=N) is documented, estimate_from_file forwards it, and _dependence_diagnostics reads it and subsamples above it — but the call inside estimate omitted the argument, so every run used the 2500 default. Asking for a cheaper diagnostic on a large frame set changed nothing.

ruff's ARG001 found it: an accepted, documented, unread parameter. Same shape as the ci_level bug in #12. Two regression tests, both watched failing with the fix reverted:

1 failed, 1 passed   <- fix reverted
2 passed             <- fix restored

CSV is no longer the interface

estimate_from_csvestimate_from_file, dispatching on the suffix: .parquet/.pq, or .csv/.tsv with optional compression. A new read_frames exposes the reader on its own.

Parquet is what the docs show now, and the reason is not taste: CSV carries no types, so a count column with one missing value comes back as float64 and timestamp comes back as a string something downstream has to guess the format of. Two tests pin this — one asserts the formats give the same estimate to twelve places, the other asserts the dtype difference is real.

No compatibility alias. The package has never been published (PyPI 404s), so nothing imports the old name.

The standard

CI, docs, release and Dependabot now call gojiplus/py-canon reusable workflows instead of carrying copies; python-publish.yml is superseded by release.yml. ruff at the fleet configuration, pyright instead of mypy, pydoclint on docstrings, plus CHANGELOG.md, CITATION.cff, pre-commit and zizmor.

That started at 326 ruff findings. Most were the config landing wrong rather than real problems:

  • per-file-ignores pointed at tests/**, but the tests live at geoinference/tests/** — so nothing was exempt. Fixing the path, and adding PT009/PT027 because the suite is unittest.TestCase throughout and rewriting 140 assertions buys nothing, cleared 235.
  • allowed-confusables = ["×"]: the codebase writes grid dimensions as 100 × 100, which is what the notation means.
  • T201 exempted for io.py and simulate.py, whose printed output is their interface.

The residue was real and is fixed: nine missing docstrings, two missing Raises sections, a lambda-as-def pair, seven over-long lines, and a test that unpacked ratio_ci.wild without checking it wasn't None.

The Python floor rises to 3.12 with the fleet. Never published, so nothing depended on 3.11.

Verification

preen check --strict   21 issues -> exit 0   (one info-level flat-layout note left)
pytest                 76 passed, 1 skipped, 6 subtests
ruff check / format    clean, 19 files
pyright                0 errors, 0 warnings
pydoclint              no violations
twine check            both artifacts PASSED
coverage               84%, CI floor set to 80

Also merged ahead of this: #12 (the ci_level fix, independently re-verified) and #5/#6/#7/#16. Note that #5/#6/#7 bumped actions in ci.yml/docs.yml, which this PR replaces with reusable-workflow callers — so those bumps are now moot, but merging them cost nothing and kept the queue clean.

🤖 Generated with Claude Code

https://claude.ai/code/session_01YDg9BEDJgFhsixKFjAquNm

Three things, all of which had to happen together because the standard's
linters are what surfaced the first.

A bug. `estimate(max_dependence_points=N)` did nothing. The argument is
documented, `estimate_from_file` forwards it, and the diagnostics
function reads it and subsamples above it -- but the call inside
`estimate` omitted it, so every run used the 2500 default. ruff's ARG001
found it: an accepted, documented, unread parameter. The same shape as
the ci_level bug in #12. Two regression tests, watched failing.

CSV is no longer the interface. `estimate_from_csv` becomes
`estimate_from_file`, dispatching on the suffix: .parquet/.pq, or
.csv/.tsv with optional compression. `read_frames` exposes the reader on
its own. Parquet is what the docs show now, because CSV carries no
types: a count column with one missing value comes back float, and
timestamp comes back a string something downstream has to parse. Tests
assert the two formats agree to twelve places and pin the dtype
difference.

The standard itself. CI, docs, release and Dependabot call py-canon's
reusable workflows; python-publish.yml is superseded by release.yml.
ruff at the fleet configuration, pyright instead of mypy, pydoclint on
docstrings. That took the lint count from 326 to zero -- mostly by
pointing per-file-ignores at geoinference/tests/ rather than the tests/
the template assumes, allowing the multiplication sign the codebase uses
for grid dimensions, and exempting the two modules whose printed output
is their interface. The rest was real: nine missing docstrings, two
missing Raises sections, and a test that unpacked a possibly-None
interval.

The Python floor rises to 3.12 with the fleet. The package has never
been published, so nothing depended on 3.11.

preen check --strict: 21 issues -> exit 0, one info-level note about the
flat layout left.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YDg9BEDJgFhsixKFjAquNm
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

CI caught two things the local run could not.

pyright on the runner resolves `geo_sampling`, which this machine cannot,
so it type-checked attribute access that locally stopped at an
unresolved import. `sample_points` calls
`geo_sampling.sample_roads_for_region` and `geo_sampling.RoadSampler`.
Neither exists. `geo_sampling` exports nothing at its top level -- it is
two CLI scripts, `geo_roads` and `sample_roads`, whose modules expose
`main` and `write_csv`. So the function raises AttributeError on the
first call, and always would have. Deleted, along with the example's
`--live` flag that was its only caller. Building a universe from a roads
file with `points_from_roads` is what the example and the tests already
do.

The docs job runs doctests. The `>>>` blocks in the package docstring and
in `estimate` referenced a `df` that was never defined, so they failed
the moment anything executed them. They now build their own frame and
assert a value: four frames, 14 women of 40 people, ratio 0.35.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YDg9BEDJgFhsixKFjAquNm
@soodoku
soodoku merged commit d80324f into main Aug 20, 2026
12 checks passed
@soodoku
soodoku deleted the adopt-py-canon branch August 20, 2026 22:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant