Skip to content

build: ground sub-package packaging in current uv/hatch/setuptools-scm guidance - #363

Closed
d-v-b wants to merge 5 commits into
chore/subpackage-standalone-pyprojectfrom
chore/subpackage-packaging-practices
Closed

d-v-b wants to merge 5 commits into
chore/subpackage-standalone-pyprojectfrom
chore/subpackage-packaging-practices

Conversation

@d-v-b

@d-v-b d-v-b commented Sep 24, 2026

Copy link
Copy Markdown
Owner

🤖 AI text below 🤖

Stacked on #361: the base is that branch, so this diff shows only these changes. Merge #361 first.

Summary

I checked our monorepo packaging against the current docs for uv (0.12.18), Hatch (1.18.1) / hatchling (1.32.4), setuptools-scm (10.3.4), hatch-vcs (0.5.0), and the PyPA sdist spec. This PR fixes the three gaps that were worth fixing.

1. Test each sdist on its own before release (ci:)

hatch build builds the wheel from the source tree, so nothing ever used the sdist. The first version of zarr-developers#4229 produced sdists that pip refuses to unpack (a symlink pointing outside the archive), and the release workflows would have uploaded them. Each release workflow's test_artifacts job now:

  • builds a wheel from the sdist with pip (pip wheel), which unpacks under the sdist format rules: no links out of the archive;
  • runs just test and just docs-check from the unpacked sdist, with no repo checkout.

The job has no checkout, so zarr-http-server resolves zarr from PyPI there. That also covers the zarr>=3.1 floor, which repo runs never check: uv run --with-editable versions are "respected regardless of the project's requirements".

2. Replace deprecated git_describe_command (build:)

setuptools-scm deprecates git_describe_command in favour of scm.git.describe_command. The DeprecationWarning is swallowed during builds. This moves the root and all three sub-packages to the new key and adds setuptools-scm>=9 to build-system.requires. hatch-vcs passes raw-options to setuptools_scm.get_version(), which only accepts scm from 9.0; 8.4, the version the docs name, was never released.

  • tag.prefix is setuptools-scm's documented option for monorepos, but hatch-vcs 0.5.0 can't reach it (get_version() takes no tag argument).
  • The remaining tag_regex deprecation warning comes from hatch-vcs's own translation of tag-pattern, so we can't fix it from our config.
  • ⚠ Anyone building zarr from source now needs setuptools-scm>=9; e.g. the conda-forge recipe's host requirements.

3. Commit zarr-metadata's uv.lock

.gitignore excluded it ("a library, not an app"). uv's docs say uv.lock "should be checked into version control", with no exception for libraries, and the other two sub-packages already commit theirs.

Checked, no change needed

Verification

Check Result
hatch version for root / zarr-metadata / zarr-indexing / zarr-http-server, before vs after identical (3.4.1.dev28+g5d4092a2b, 0.5.1.dev80, 0.2.2.dev91, 0.1.1.dev95), each +1 commit after committing
setuptools-scm 8.2.0 / 8.3.1 with scm= TypeError, hence >=9
new pip step on the old broken zarr-developers#4229 sdist fails: "would link to … outside the destination"
new pip step on sdists from this branch (hatch 1.16.5, as in CI) all three build
just test from unpacked sdists metadata 595 passed; indexing 1547 passed / 207 skipped (no zarr); http-server 212 passed / 9 skipped
just docs-check from unpacked sdists all three build under --strict
zizmor, GitHub workflow schema clean

I haven't run the release workflows: on the fork, workflow_dispatch would also trigger the TestPyPI upload job.

Once zarr-developers#4229 lands, zarr-metadata's new uv.lock needs a uv lock for zarr-developers#4229's aligned docs pins.

🤖 Generated with Claude Code

d-v-b added a commit that referenced this pull request Sep 24, 2026
Assisted-by: ClaudeCode:claude-opus-5-5
d-v-b added a commit that referenced this pull request Sep 25, 2026
Assisted-by: ClaudeCode:claude-opus-5-5
@d-v-b
d-v-b force-pushed the chore/subpackage-packaging-practices branch from 940f104 to c027cac Compare September 25, 2026 06:25
@d-v-b
d-v-b force-pushed the chore/subpackage-standalone-pyproject branch from 5d4092a to 0560a62 Compare September 25, 2026 06:25
…uv.lock

setuptools-scm deprecates the top-level `git_describe_command` in favour of
`scm.git.describe_command` (a DeprecationWarning builds swallow). Move the
root and all three sub-packages to the new key; hatch-vcs passes raw-options
to setuptools_scm.get_version, which accepts `scm` from setuptools-scm 9.0
(8.4, which the docs name, was never released), so require
setuptools-scm>=9 at build time. Computed versions are unchanged.

`tag.prefix`, setuptools-scm's documented monorepo option, is not reachable
through hatch-vcs 0.5.0: get_version() takes no `tag` argument.

uv's docs say uv.lock "should be checked into version control", with no
library exception, and the other two sub-packages already commit theirs.
Stop ignoring zarr-metadata's.

Assisted-by: ClaudeCode:claude-opus-5-5
`hatch build` builds the wheel from the source tree, so nothing exercised
the sdist: zarr-developers#4229's first cut shipped sdists whose symlinks pip refuses to
unpack, and the release workflows would have uploaded them. The
test_artifacts job now builds a wheel from the sdist with pip, then runs
the package's `just test` and `just docs-check` from the unpacked sdist.
The job has no checkout, so zarr (for zarr-http-server) resolves from
PyPI, which also covers the `zarr>=3.1` floor that repo runs, overlaying
in-repo zarr, never check.

Assisted-by: ClaudeCode:claude-opus-5-5
Assisted-by: ClaudeCode:claude-opus-5-5
Assisted-by: ClaudeCode:claude-opus-5-5
@d-v-b
d-v-b force-pushed the chore/subpackage-standalone-pyproject branch from 0560a62 to 1d197cf Compare September 25, 2026 06:26
@d-v-b
d-v-b force-pushed the chore/subpackage-packaging-practices branch from c027cac to 00d6643 Compare September 25, 2026 06:26
@d-v-b

d-v-b commented Sep 25, 2026

Copy link
Copy Markdown
Owner Author

🤖 AI text below 🤖

Superseded by zarr-developers#4407.

@d-v-b d-v-b closed this Sep 25, 2026
d-v-b added a commit that referenced this pull request Sep 25, 2026
…m guidance (zarr-developers#4407)

* chore: keep sub-package pyproject.toml standalone; overlay in-repo zarr from just

The packages reached in-repo zarr two different ways: zarr-http-server
pinned it with `[tool.uv.sources] zarr = { path = "../.." }` in
pyproject.toml, and zarr-indexing ran its tests in the repo root's
environment (`uv run --project ../..`). Both reach outside the package, so
neither package's unpacked sdist could run its own `just test`.

Now every package's pyproject.toml stands alone, and the justfiles add
the in-repo zarr as an editable overlay (`--with-editable ../..`) only
when ../.. is the repository. From an sdist there is no overlay:
zarr-http-server resolves zarr from PyPI, and zarr-indexing's zarr suites
skip. zarr-metadata needs no zarr and is unchanged.

zarr-http-server's uv.lock now resolves zarr from PyPI, and zarr-indexing
CI syncs the package environment instead of the root one.

Assisted-by: ClaudeCode:claude-opus-5-5

* chore: add changelog fragments for zarr-developers#4406

Assisted-by: ClaudeCode:claude-opus-5-5

* build: replace deprecated git_describe_command; commit zarr-metadata uv.lock

setuptools-scm deprecates the top-level `git_describe_command` in favour of
`scm.git.describe_command` (a DeprecationWarning builds swallow). Move the
root and all three sub-packages to the new key; hatch-vcs passes raw-options
to setuptools_scm.get_version, which accepts `scm` from setuptools-scm 9.0
(8.4, which the docs name, was never released), so require
setuptools-scm>=9 at build time. Computed versions are unchanged.

`tag.prefix`, setuptools-scm's documented monorepo option, is not reachable
through hatch-vcs 0.5.0: get_version() takes no `tag` argument.

uv's docs say uv.lock "should be checked into version control", with no
library exception, and the other two sub-packages already commit theirs.
Stop ignoring zarr-metadata's.

Assisted-by: ClaudeCode:claude-opus-5-5

* ci: test each sub-package sdist standalone before release

`hatch build` builds the wheel from the source tree, so nothing exercised
the sdist: zarr-developers#4229's first cut shipped sdists whose symlinks pip refuses to
unpack, and the release workflows would have uploaded them. The
test_artifacts job now builds a wheel from the sdist with pip, then runs
the package's `just test` and `just docs-check` from the unpacked sdist.
The job has no checkout, so zarr (for zarr-http-server) resolves from
PyPI, which also covers the `zarr>=3.1` floor that repo runs, overlaying
in-repo zarr, never check.

Assisted-by: ClaudeCode:claude-opus-5-5

* chore: add changelog fragment for #363

Assisted-by: ClaudeCode:claude-opus-5-5

* chore(zarr-metadata): relock for the docs pins aligned in zarr-developers#4229

Assisted-by: ClaudeCode:claude-opus-5-5

* chore: renumber changelog fragment for zarr-developers#4407

Assisted-by: ClaudeCode:claude-opus-5-5
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant