Conversation
…s their container A structured data type resolved its field data types with the default data type registry, whatever registry resolved the structured data type itself, so a DataTypeRegistry other than the default could not supply field data types. - DTypeResolver (exported from zarr.dtype) is the signature of DataTypeRegistry.match_json. - HasNestedDTypes is a mix-in for data types that contain data types. A registry creates them with the _from_json_nested hook, passing its own match_json as the resolver. Other data types are created with from_json as before, so data types that override from_json keep working. - Structured and Struct are HasNestedDTypes, and their from_json takes an optional resolver (the default registry when omitted). - get_data_type_from_json, get_data_type_from_native_dtype, parse_dtype and parse_data_type take an optional registry (the default registry when omitted). Assisted-by: ClaudeCode:claude-opus-5-5
zarr.registry collects data type entry points into DataTypeRegistry._lazy_load_list, but nothing in src called _lazy_load, so data types registered through the "zarr.data_type" entry point group never loaded. get, match_dtype and match_json now load them first, as the other registries do in their getters. _lazy_load imports each entry point once instead of twice. The test fixture that clears pending entry points now clears the data type registry's too, since _collect_entrypoints does not return it, and the entry point test no longer loads them by hand. Assisted-by: ClaudeCode:claude-opus-5-5
Assisted-by: ClaudeCode:claude-opus-5-5
Replaces the DTypeResolver callable with a context object, as mature
decoders do (Jackson's DeserializationContext, Gson's Gson instance,
PyYAML's loader), so it can grow without changing the hook's signature.
- DTypeContext (exported from zarr.dtype) holds the registry, the Zarr
format and the path of the data type within the data types that contain
it. Composites resolve each contained data type with
context.child(name).resolve(json). The Zarr format is part of the context
because contained data types always share their container's format.
- HasNestedDTypes._from_json_nested takes the context.
- Structured.from_json / Struct.from_json take registry= like the other
entry points, instead of a resolver.
- NestedDataTypeValidationError (a DataTypeValidationError) reports a
contained data type that matches nothing, with its path. The registry
re-raises it instead of trying other data types, since the JSON has the
shape of the containing data type. This restores a specific error for an
invalid struct field (e.g. "... matches {'name': '|i9', ...} at path
['a']"), which #351 had reduced to a mismatch of the whole struct.
Assisted-by: ClaudeCode:claude-opus-5-5
…state DTypeContext mixed two things: which extensions a read may resolve (the same for a whole read, chosen by the caller) and where the read has got to (changing at every nested step). That does not extend to other kinds of extension: a codec read that meets a data type (cast_value) must hand the same extensions to a data type read. - zarr.registry.Context is the set of extensions, by kind. It holds only data_types for now; codecs, chunk key encodings and chunk grids can be added as fields with defaults. Context.default() refers to the default registries. This mirrors zarr-metadata's Context, passed as context=. - Reading (zarr.core.context) is the state of a read: the context, the Zarr format and the location in the document. Composites read contained extensions with reading.at(*keys).resolve_data_type(json). - The location is the JSON location in the document, reported as a JSON Pointer, so it stays unambiguous once codecs nest data types. A Zarr V2 data type is located by its name as written in the document. - The containing data type decides that a failure is nested: Structured raises NestedDataTypeValidationError for a field that matches nothing, since a non-empty location no longer means "inside another data type" once metadata parsing starts reads at /data_type. - The dtype entry points and Structured.from_json take context= instead of registry=. Assisted-by: ClaudeCode:claude-opus-5-5
… request It was merged under its fork pull request number (351), which towncrier would link to the wrong pull request. Assisted-by: ClaudeCode:claude-opus-5-5
Assisted-by: ClaudeCode:claude-opus-5-5
d-v-b
force-pushed
the
feat/open-with-context
branch
from
September 23, 2026 15:20
fede738 to
0013cbb
Compare
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #4388 +/- ##
==========================================
+ Coverage 94.26% 94.28% +0.01%
==========================================
Files 92 93 +1
Lines 12981 13051 +70
==========================================
+ Hits 12237 12305 +68
- Misses 744 746 +2
🚀 New features to boost your workflow:
|
A Resolver is a Context positioned in a document; it resolves the extensions there. The pair mirrors referencing.Registry and referencing.Resolver in python-jsonschema. "Reading" read as a verb. Assisted-by: ClaudeCode:claude-opus-5-5
d-v-b
force-pushed
the
feat/open-with-context
branch
from
September 23, 2026 17:03
0013cbb to
0ca6d6f
Compare
ZDType._from_json_resolved(data, *, resolver) is the one way a registry creates a data type. Its default ignores the resolver and calls from_json in the resolver's Zarr format, so data types that override from_json keep working; a data type that contains others (Structured, Struct) overrides it to resolve them. This replaces the HasNestedDTypes mix-in and the registry's issubclass branch: every type receives the context and leaves ignore it, as in pydantic's __get_pydantic_core_schema__(source, handler), serde's Deserialize, and zarr-metadata's resolve(data, context). Assisted-by: ClaudeCode:claude-opus-5-5
Metadata reads now take the extensions they resolve from a caller-chosen zarr.registry.Context instead of always the default registries. - ArrayV2Metadata.from_dict, ArrayV3Metadata.from_dict, ConsolidatedMetadata.from_dict, GroupMetadata.from_dict and parse_array_metadata take context=. Data types are read at /dtype (v2) or /data_type (v3), so errors carry their location in the document. - AsyncGroup has a context field (default Context.default(); excluded from equality and repr). The metadata of its members is read with it through getitem, members and consolidated metadata, and groups reached through it or created in it (create_group, create_hierarchy) share it. Arrays do not hold one: their metadata is fully read when they are opened. - zarr.open, zarr.open_array, zarr.open_group, zarr.open_consolidated (sync and async), Array.open / from_dict, AsyncArray.open / from_dict, Group.open / from_store, AsyncGroup.open / from_store / from_dict and get_node (both) take context=. Array creation still resolves with the default registries. Assisted-by: ClaudeCode:claude-opus-5-5
Assisted-by: ClaudeCode:claude-opus-5-5
Assisted-by: ClaudeCode:claude-opus-5-5
d-v-b
force-pushed
the
feat/open-with-context
branch
from
September 23, 2026 17:34
0ca6d6f to
fa37017
Compare
d-v-b
marked this pull request as ready for review
September 27, 2026 07:48
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🤖 AI text below 🤖
Note
Parked. #4387 was narrowed to internal changes only: it no longer exports
Context, adds nocontext=keywords, and has no publicNestedDataTypeValidationError. This PR makes the context public throughzarr.open,open_array,open_group,open_consolidatedand the class and metadata constructors. It stays parked until there's a concrete caller for choosing a read's extensions, or until codecs are part of the context and the public shape is settled. Its branch is built on the earlier, public version of #4387 and will be rebuilt on the internal one before it's unparked; the description below describes that earlier version.Depends on #4387, and includes its commits; only the commits from
feat: open arrays and groups with a Contextonwards (it and its changelog fragment) are new here. Merge #4387 first; this branch will be rebased ontomainafterwards.Problem
#4387 lets data type resolution take a
Context, but nothing that opens an array or group passes one. Metadata parsing, group member reads and consolidated metadata all still resolve through the default registries. So there was no way to open data with extensions other than the global ones, for example opening NCZarr data with or withoutNCZarrChar(#4386), or reading an array whose data type is registered in a local registry.Change
ArrayV2Metadata.from_dict,ArrayV3Metadata.from_dict,ConsolidatedMetadata.from_dict,GroupMetadata.from_dictandparse_array_metadatatakecontext=. Data types are read at/dtype(v2) or/data_type(v3), so a failure names its place in the document, e.g.No Zarr data type found that matches 'test.byte' at /data_type.AsyncGrouphas acontextfield (defaultContext.default(), excluded from equality and repr). Its members' metadata is read with that context throughgetitem,membersand consolidated metadata. Groups reached through it, and groups created in it (create_group,create_hierarchy), share it.context=onzarr.open,open_array,open_groupandopen_consolidated(sync and async); onArray.open/from_dictandAsyncArray.open/from_dict; onGroup.open/from_storeandAsyncGroup.open/from_store/from_dict; and onget_node(both). The default everywhere isContext.default(), so behaviour withoutcontext=is unchanged.Designed to grow
Contextholds onlydata_typesfor now. Codecs are next. The v3 codec registry (with itszarr.configimplementation selection as the default), chunk key encodings, chunk grids and v2 numcodecs can each become a field with a default. Adding one changes no signature in this PR. The internalResolver(context, Zarr format, location) is what a codec read will hand to a data type read insidecast_value, and what sharding will use for its inner codecs.Not in this PR
create_arrayand friends still resolve JSON data types with the default registries.Codec.from_dict(data), overridden by sharding andcast_value./data_type, not/consolidated_metadata/metadata/<name>/data_type.Tests
tests/test_context.py:test.bytein v3, a class claiming<u1in v2) must resolve to that class. The routes areopen_array,zarr.open, groupgetitem, getitem two groups deep,members(max_depth=None), and consolidatedgetitem.create_groupin a group opened with a context reads its members with it.test.byteat/data_type.🤖 Generated with Claude Code