Skip to content

Avoid duplicate metadata fetches when zarr.open() falls back from array to group - #192

Draft
d-v-b wants to merge 9 commits into
mainfrom
claude/open-single-probe
Draft

d-v-b wants to merge 9 commits into
mainfrom
claude/open-single-probe

Conversation

@d-v-b

@d-v-b d-v-b commented Jun 12, 2026 •

Copy link
Copy Markdown
Owner

🤖 AI text below 🤖

zarr.open reuses metadata buffers retained by its array probe when falling back to opening a group. Buffer reuse currently applies when zarr_format is inferred; explicit-format probes do not retain these buffers, even where their reads overlap with group opening.

The fetch-count regression covers 30 combinations of read/update modes, root/nested paths, V2/V3 metadata, and disabled/required/custom-key consolidation. It checks returned metadata and at most one read per key. Audit validation: hatch run test.py3.12-minimal:pytest tests/test_api.py -k no_duplicate_fetches -q — 30 passed, 367 deselected.

Docstrings now distinguish returned missing-array/node-type/format-selection errors from JSON and store failures that propagate. Previous 480-scenario differential and broader suite results are historical audit records, not independently reproduced results from this pass.

dependabot Bot and others added 7 commits May 31, 2026 19:28
…#176)

Bumps the actions group with 8 updates in the / directory:

| Package | From | To |
| --- | --- | --- |
| [prefix-dev/setup-pixi](https://github.com/prefix-dev/setup-pixi) | `0.9.5` | `0.9.6` |
| [codecov/codecov-action](https://github.com/codecov/codecov-action) | `6.0.0` | `6.0.1` |
| [github/issue-metrics](https://github.com/github/issue-metrics) | `4.2.2` | `4.2.7` |
| [j178/prek-action](https://github.com/j178/prek-action) | `2.0.3` | `2.0.4` |
| [actions/upload-artifact](https://github.com/actions/upload-artifact) | `7.0.0` | `7.0.1` |
| [actions/download-artifact](https://github.com/actions/download-artifact) | `7.0.0` | `8.0.1` |
| [pypa/gh-action-pypi-publish](https://github.com/pypa/gh-action-pypi-publish) | `1.13.0` | `1.14.0` |
| [zizmorcore/zizmor-action](https://github.com/zizmorcore/zizmor-action) | `0.5.3` | `0.5.6` |



Updates `prefix-dev/setup-pixi` from 0.9.5 to 0.9.6
- [Release notes](https://github.com/prefix-dev/setup-pixi/releases)
- [Commits](prefix-dev/setup-pixi@1b2de7f...5185adf)

Updates `codecov/codecov-action` from 6.0.0 to 6.0.1
- [Release notes](https://github.com/codecov/codecov-action/releases)
- [Changelog](https://github.com/codecov/codecov-action/blob/main/CHANGELOG.md)
- [Commits](codecov/codecov-action@57e3a13...e79a696)

Updates `github/issue-metrics` from 4.2.2 to 4.2.7
- [Release notes](https://github.com/github/issue-metrics/releases)
- [Commits](github-community-projects/issue-metrics@c9e9838...1e38d5e)

Updates `j178/prek-action` from 2.0.3 to 2.0.4
- [Release notes](https://github.com/j178/prek-action/releases)
- [Commits](j178/prek-action@6ad8027...bdca6f1)

Updates `actions/upload-artifact` from 7.0.0 to 7.0.1
- [Release notes](https://github.com/actions/upload-artifact/releases)
- [Commits](actions/upload-artifact@v7...043fb46)

Updates `actions/download-artifact` from 7.0.0 to 8.0.1
- [Release notes](https://github.com/actions/download-artifact/releases)
- [Commits](actions/download-artifact@v7...3e5f45b)

Updates `pypa/gh-action-pypi-publish` from 1.13.0 to 1.14.0
- [Release notes](https://github.com/pypa/gh-action-pypi-publish/releases)
- [Commits](pypa/gh-action-pypi-publish@v1.13.0...cef2210)

Updates `zizmorcore/zizmor-action` from 0.5.3 to 0.5.6
- [Release notes](https://github.com/zizmorcore/zizmor-action/releases)
- [Commits](zizmorcore/zizmor-action@b1d7e1f...5f14fd0)

---
updated-dependencies:
- dependency-name: prefix-dev/setup-pixi
  dependency-version: 0.9.6
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: actions
- dependency-name: codecov/codecov-action
  dependency-version: 6.0.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: actions
- dependency-name: github/issue-metrics
  dependency-version: 4.2.7
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: actions
- dependency-name: j178/prek-action
  dependency-version: 2.0.4
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: actions
- dependency-name: actions/upload-artifact
  dependency-version: 7.0.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: actions
- dependency-name: actions/download-artifact
  dependency-version: 8.0.1
  dependency-type: direct:production
  update-type: version-update:semver-major
  dependency-group: actions
- dependency-name: pypa/gh-action-pypi-publish
  dependency-version: 1.14.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: actions
- dependency-name: zizmorcore/zizmor-action
  dependency-version: 0.5.6
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: actions
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…ay to group

`zarr.open()` with `zarr_format=None` probed for array metadata (3 GETs:
zarr.json, .zarray, .zattrs) and, on a group path, then re-probed via
`open_group()` (4 GETs: zarr.json, .zgroup, .zattrs, .zmetadata), fetching
zarr.json and .zattrs twice.

This refactors the array probe (`_probe_array_metadata`) to return the
buffers it fetched, and threads them into the group-open path via a private
`_pre_fetched_metadata` parameter on `open_group`/`AsyncGroup.open`. The
group open now reuses the already-fetched zarr.json and .zattrs documents
instead of re-fetching them. v2/v3 precedence, the both-formats warning,
consolidated-metadata handling, and all exception types/messages are
preserved unchanged.

For a v3 group via `open()`: 7 GETs (2 duplicated) -> 5 GETs (0 duplicated).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Refresh current main and cover metadata reuse across read modes, nested paths, and custom consolidated keys. Clean up API test typing so focused checks pass.

Assisted-by: Codex:GPT-6

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant