Repository navigation
test: check the XML models against the oddsfeedschema XSDs - #61
Merged
Merged
Conversation
The feed and REST XML models are written by hand and nothing tied them to the schema, so an attribute the producer starts sending — or one the schema gains — was silently dropped until someone noticed. Generating the structs from the XSDs was tried and rejected: the Go generators (xgen, xsdgen) cannot resolve the schema's namespace-less xs:includes, and even a working one would lose the hand-tuned parts (epoch-millisecond timestamps, typed enums, pointer-typed optional attributes, the shared outcome struct). A test is cheaper than a second layer plus an adapter. internal/schemacheck reduces both sides to the same structural shape — attribute names, child elements, text content — and compares them root element by root element: - a minimal XSD reader for the subset oddsfeedschema uses (chameleon includes into one symbol table per wire, attributeGroups, complex and simple content extensions, inline types, choice/sequence); - a reflector that reads encoding/xml struct tags the way encoding/xml does (attr, a>b paths, chardata, embedded structs, pointers/slices, TextUnmarshaler types as leaves). Deviations are kept in a ledger with a reason each. A deviation not in the ledger fails the build; so does a ledger entry nothing matches any more, so the ledger is always exactly the current drift. oddsfeedschema stays the single source of truth: nothing is vendored. TestMain downloads the repository archive at test time — ref main by default, ODDSFEEDSCHEMA_REF to pin a branch/tag/commit, ODDSFEEDSCHEMA_DIR to use a local checkout — so a schema change upstream fails the next gosdk build, which is the point. Without network the schema tests skip locally and fail in CI. The check matches the schema layout of oddsfeedschema PR #18 (schema/common, bet_stop, error); against the pre-#18 main it fails on the missing schema/common directory, so #18 must land first. What the first run found, now ledgered with reasons: - schema declares, SDK drops: sport_event type / start_time_tbd (no SDK surface yet), the declared-but-never-emitted Betradar-heritage aggregate_* / status_code, market@void_reason, player_profile generated_at; - SDK decodes, schema does not declare: ref_id / event_ref_id / extended_specifiers everywhere (legacy wire fields, never sent), fields the shared Go types carry into contexts that lack them (feedXML.Outcome, MarketWithOutcome, apiXML.Sport), and three the SDK actively consumes — <statistics>, <category>, <reference_ids> — which the schema owners need to confirm or declare. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
dsaiko
force-pushed
the
test/xsd-conformance
branch
from
September 8, 2026 08:14
36bb410 to
084bc49
Compare
…ind type names Review of the check found its compare step discarding a value both sides compute: the XSD reader marks simple types, simpleContent and mixed content as text, the Go reflector marks chardata/cdata/innerxml fields as text, and compare never looked at either — so a declared text slot with no Go field for it (or the reverse) was invisible to the very gate whose job is to notice such drift. compare now reports the mismatch as "<path>#text". On oddsfeedschema this surfaces three declared-but- never-sent text slots — the rollback messages' <market>, and <void_reason> plus its <param> — which are ledgered as such. The loader documented that duplicate named types across the included files are an error but only enforced it for roots, complexTypes and attributeGroups; simpleTypes were inserted unchecked, and because a simple type is resolved before a complex one, a name declared as both would have silently flattened the element into a text leaf. Both cases are now refused at load time. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
oddsfeedschema#18 landed on 2026-09-09 (ecd0afa), so the conformance tests now resolve schema/common, bet_stop.xsd and error.xsd from the default main ref. The merged schema/ tree is byte-identical to the branch head the ledger was written against, so nothing in the check changes; this commit only re-triggers CI against the merged upstream. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The skip reasons named a third-party schema as the origin of the attributes the Go models decode but no producer sends. This repository is public and that provenance does not belong in it; "legacy" says everything the reason needs to say. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Merging this branch will increase overall coverage
Coverage by fileChanged files (no unit tests)
Please note that the "Total", "Covered", and "Missed" counts above refer to code statements instead of lines of code. The value in brackets refers to the test coverage of that file in the old version of the code. Changed unit test files
|
yung-madamm
approved these changes
Sep 14, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
The feed and REST XML models (
internal/feed/xml,internal/api/xml) are written by hand and nothing tied them to oddsfeedschema, so an attribute the producer starts sending — or one the schema gains — was silently dropped until someone noticed. oddsfeedschema#18 puts a conformance gate in front of the producer; this is the SDK's counterpart.Generating the structs from the XSDs was evaluated first and rejected: neither Go generator (xgen, xsdgen) resolves the schema's namespace-less
xs:includes, and a working one would still lose the hand-tuned parts — epoch-millisecond timestamps, typed enums, pointer-typed optional attributes, the shared outcome struct. The model is ~1900 lines; a test is cheaper than a second layer plus an adapter.What
internal/schemacheckreduces both sides to one structural shape — attribute names, child elements, text content — and compares them root element by root element.attributeGroups, complex/simple content extensions, inline types,choice/sequence. Duplicate type names across files — simple or complex, and a name used for both — are an error (nothing else would catch a shadowing).#textin ledger paths): a simple type,simpleContentormixedelement on the schema side must meet achardatafield on the Go side and vice versa. This surfaces three declared-but-never-sent text slots (rollback_bet_cancel/rollback_bet_settlement<market>,void_reasons<void_reason>and its<param>), ledgered as such.encoding/xmltags the wayencoding/xmldoes:attr,a>bpaths,chardata, embedded structs, pointers/slices,TextUnmarshalertypes as leaves (soutils.Timestampdoes not explode intotime.Time's fields).TestMaindownloads the oddsfeedschema archive at test time and readsschema/from it. Refmainby default;ODDSFEEDSCHEMA_REFpins a branch/tag/commit,ODDSFEEDSCHEMA_DIRuses a local checkout (offline, or to try a schema branch). Without network the schema tests skip locally and fail in CI (CI=true), so a broken download cannot pass as green.Upstream dependency: satisfied
The check follows the layout oddsfeedschema#18 introduces (
schema/common,bet_stop.xsd,error.xsd). #18 was merged on 2026-09-09 (ecd0afa), so no merge order applies any more: the tests run against oddsfeedschemamainby default and pass. The mergedschema/tree is byte-identical to the branch head this PR was developed against, so the ledger below is exactly the drift againstmain.Verified after the merge: the gate bites in both directions against
main— deleting one ledger entry surfaces 29 unledgered deviations, adding a bogus one is reported as stale. PinningODDSFEEDSCHEMA_REFto a branch, a full commit SHA or a short SHA all work; an unreachable ref skips locally and fails underCI.What the first run found
All ledgered with reasons; nothing here changes SDK behaviour.
Schema declares, SDK drops
sport_event/fixture@type,@start_time_tbd— producer sends them, the SDK has no API surface for them yetsport_event_status@status_code,@aggregate_*— declared Betradar heritage, never emitted (per the schema's own comment)bet_cancel/market@void_reason(SDK readsvoid_reason_id),player_profile@generated_atSDK decodes, schema does not declare
ref_id/event_ref_id/sport_event_ref_id/extended_specifierseverywhere — legacy wire fields no producer sends; candidates for removal from the modelsfeedXML.Outcome(odds attrs on settlement outcomes and vice versa),MarketWithOutcome(cancel/odds attrs across markets),apiXML.Sport(icon_pathon every plain<sport>)<statistics>on the feedsport_event_status,<category>under tournaments and competitor profiles,<reference_ids>on sport events and tournaments. Either the schema is behind the producer or the SDK carries dead features — for the schema owners to settle. Ledgered as "schema owners to confirm".tournament_scheduleis the one schema root the SDK has no type for (endpoint not called); recorded as such.Tests
TestSchemaCoverage(feed + rest against the fetched schema)TestXSDReader_*,TestGoShape_ReadsTagsLikeEncodingXML,TestCompareAndReconcileODDSFEEDSCHEMA_REF=feature/schema-producer-conformanceand withODDSFEEDSCHEMA_DIR=<local checkout>; unreachable ref → skip locally, fail withCI=1go test -race,make lint(pinned v2.12.1) 0 issues🤖 Generated with Claude Code