Skip to content

test: check the XML models against the oddsfeedschema XSDs - #61

Merged
dsaiko merged 4 commits into
mainfrom
test/xsd-conformance
Sep 14, 2026
Merged

dsaiko merged 4 commits into
mainfrom
test/xsd-conformance

Conversation

@dsaiko

@dsaiko dsaiko commented Sep 8, 2026 •

Copy link
Copy Markdown
Contributor

Why

The feed and REST XML models (internal/feed/xml, internal/api/xml) are written by hand and nothing tied them to oddsfeedschema, so an attribute the producer starts sending — or one the schema gains — was silently dropped until someone noticed. oddsfeedschema#18 puts a conformance gate in front of the producer; this is the SDK's counterpart.

Generating the structs from the XSDs was evaluated first and rejected: neither Go generator (xgen, xsdgen) resolves the schema's namespace-less xs:includes, and a working one would still lose the hand-tuned parts — epoch-millisecond timestamps, typed enums, pointer-typed optional attributes, the shared outcome struct. The model is ~1900 lines; a test is cheaper than a second layer plus an adapter.

What

internal/schemacheck reduces both sides to one structural shape — attribute names, child elements, text content — and compares them root element by root element.

  • XSD reader for the subset oddsfeedschema uses: chameleon includes merged into one symbol table per wire, attributeGroups, complex/simple content extensions, inline types, choice/sequence. Duplicate type names across files — simple or complex, and a name used for both — are an error (nothing else would catch a shadowing).
  • Text content is compared too (#text in ledger paths): a simple type, simpleContent or mixed element on the schema side must meet a chardata field on the Go side and vice versa. This surfaces three declared-but-never-sent text slots (rollback_bet_cancel/rollback_bet_settlement <market>, void_reasons <void_reason> and its <param>), ledgered as such.
  • Go reflector that reads encoding/xml tags the way encoding/xml does: attr, a>b paths, chardata, embedded structs, pointers/slices, TextUnmarshaler types as leaves (so utils.Timestamp does not explode into time.Time's fields).
  • Drift ledger in the test, one line per known deviation with its reason. A deviation not in the ledger fails; so does a ledger entry nothing matches any more. The ledger is therefore always exactly the current drift, and reviewable.
  • Single source of truth — nothing vendored. TestMain downloads the oddsfeedschema archive at test time and reads schema/ from it. Ref main by default; ODDSFEEDSCHEMA_REF pins a branch/tag/commit, ODDSFEEDSCHEMA_DIR uses a local checkout (offline, or to try a schema branch). Without network the schema tests skip locally and fail in CI (CI=true), so a broken download cannot pass as green.

Upstream dependency: satisfied

The check follows the layout oddsfeedschema#18 introduces (schema/common, bet_stop.xsd, error.xsd). #18 was merged on 2026-09-09 (ecd0afa), so no merge order applies any more: the tests run against oddsfeedschema main by default and pass. The merged schema/ tree is byte-identical to the branch head this PR was developed against, so the ledger below is exactly the drift against main.

Verified after the merge: the gate bites in both directions against main — deleting one ledger entry surfaces 29 unledgered deviations, adding a bogus one is reported as stale. Pinning ODDSFEEDSCHEMA_REF to a branch, a full commit SHA or a short SHA all work; an unreachable ref skips locally and fails under CI.

What the first run found

All ledgered with reasons; nothing here changes SDK behaviour.

Schema declares, SDK drops

  • sport_event / fixture @type, @start_time_tbd — producer sends them, the SDK has no API surface for them yet
  • sport_event_status @status_code, @aggregate_* — declared Betradar heritage, never emitted (per the schema's own comment)
  • bet_cancel/market@void_reason (SDK reads void_reason_id), player_profile@generated_at

SDK decodes, schema does not declare

  • ref_id / event_ref_id / sport_event_ref_id / extended_specifiers everywhere — legacy wire fields no producer sends; candidates for removal from the models
  • fields the shared Go types carry into contexts that lack them: feedXML.Outcome (odds attrs on settlement outcomes and vice versa), MarketWithOutcome (cancel/odds attrs across markets), apiXML.Sport (icon_path on every plain <sport>)
  • three the SDK actively consumes: <statistics> on the feed sport_event_status, <category> under tournaments and competitor profiles, <reference_ids> on sport events and tournaments. Either the schema is behind the producer or the SDK carries dead features — for the schema owners to settle. Ledgered as "schema owners to confirm".

tournament_schedule is the one schema root the SDK has no type for (endpoint not called); recorded as such.

Tests

  • TestSchemaCoverage (feed + rest against the fetched schema)
  • machinery pinned on tiny inputs: TestXSDReader_*, TestGoShape_ReadsTagsLikeEncodingXML, TestCompareAndReconcile
  • verified: pass with ODDSFEEDSCHEMA_REF=feature/schema-producer-conformance and with ODDSFEEDSCHEMA_DIR=<local checkout>; unreachable ref → skip locally, fail with CI=1
  • go test -race, make lint (pinned v2.12.1) 0 issues

🤖 Generated with Claude Code

The feed and REST XML models are written by hand and nothing tied them to
the schema, so an attribute the producer starts sending — or one the
schema gains — was silently dropped until someone noticed. Generating the
structs from the XSDs was tried and rejected: the Go generators (xgen,
xsdgen) cannot resolve the schema's namespace-less xs:includes, and even
a working one would lose the hand-tuned parts (epoch-millisecond
timestamps, typed enums, pointer-typed optional attributes, the shared
outcome struct). A test is cheaper than a second layer plus an adapter.

internal/schemacheck reduces both sides to the same structural shape —
attribute names, child elements, text content — and compares them root
element by root element:

- a minimal XSD reader for the subset oddsfeedschema uses (chameleon
  includes into one symbol table per wire, attributeGroups, complex and
  simple content extensions, inline types, choice/sequence);
- a reflector that reads encoding/xml struct tags the way encoding/xml
  does (attr, a>b paths, chardata, embedded structs, pointers/slices,
  TextUnmarshaler types as leaves).

Deviations are kept in a ledger with a reason each. A deviation not in
the ledger fails the build; so does a ledger entry nothing matches any
more, so the ledger is always exactly the current drift.

oddsfeedschema stays the single source of truth: nothing is vendored.
TestMain downloads the repository archive at test time — ref main by
default, ODDSFEEDSCHEMA_REF to pin a branch/tag/commit,
ODDSFEEDSCHEMA_DIR to use a local checkout — so a schema change upstream
fails the next gosdk build, which is the point. Without network the
schema tests skip locally and fail in CI.

The check matches the schema layout of oddsfeedschema PR #18
(schema/common, bet_stop, error); against the pre-#18 main it fails on
the missing schema/common directory, so #18 must land first.

What the first run found, now ledgered with reasons:
- schema declares, SDK drops: sport_event type / start_time_tbd (no SDK
  surface yet), the declared-but-never-emitted Betradar-heritage
  aggregate_* / status_code, market@void_reason, player_profile
  generated_at;
- SDK decodes, schema does not declare: ref_id / event_ref_id /
  extended_specifiers everywhere (legacy wire fields, never sent),
  fields the shared Go types carry into contexts that lack them
  (feedXML.Outcome, MarketWithOutcome, apiXML.Sport), and three the SDK
  actively consumes — <statistics>, <category>, <reference_ids> — which
  the schema owners need to confirm or declare.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@dsaiko
dsaiko force-pushed the test/xsd-conformance branch from 36bb410 to 084bc49 Compare September 8, 2026 08:14
@dsaiko dsaiko changed the title test: check the XML models against the oddsfeedschema XSDs and fixtures test: check the XML models against the oddsfeedschema XSDs Sep 8, 2026
dsaiko and others added 2 commits September 10, 2026 21:18
…ind type names

Review of the check found its compare step discarding a value both
sides compute: the XSD reader marks simple types, simpleContent and
mixed content as text, the Go reflector marks chardata/cdata/innerxml
fields as text, and compare never looked at either — so a declared text
slot with no Go field for it (or the reverse) was invisible to the very
gate whose job is to notice such drift. compare now reports the mismatch
as "<path>#text". On oddsfeedschema this surfaces three declared-but-
never-sent text slots — the rollback messages' <market>, and
<void_reason> plus its <param> — which are ledgered as such.

The loader documented that duplicate named types across the included
files are an error but only enforced it for roots, complexTypes and
attributeGroups; simpleTypes were inserted unchecked, and because a
simple type is resolved before a complex one, a name declared as both
would have silently flattened the element into a text leaf. Both cases
are now refused at load time.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
oddsfeedschema#18 landed on 2026-09-09 (ecd0afa), so the conformance
tests now resolve schema/common, bet_stop.xsd and error.xsd from the
default main ref. The merged schema/ tree is byte-identical to the
branch head the ledger was written against, so nothing in the check
changes; this commit only re-triggers CI against the merged upstream.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The skip reasons named a third-party schema as the origin of the attributes
the Go models decode but no producer sends. This repository is public and
that provenance does not belong in it; "legacy" says everything the reason
needs to say.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown

Merging this branch will increase overall coverage

Impacted Packages Coverage Δ 🤖
github.com/oddin-gg/gosdk/internal/schemacheck 83.13% (+83.13%) 🌟

Coverage by file

Changed files (no unit tests)

Changed File Coverage Δ Total Covered Missed 🤖
github.com/oddin-gg/gosdk/internal/schemacheck/compare.go 86.67% (+86.67%) 45 (+45) 39 (+39) 6 (+6) 🌟
github.com/oddin-gg/gosdk/internal/schemacheck/gomodel.go 87.30% (+87.30%) 63 (+63) 55 (+55) 8 (+8) 🌟
github.com/oddin-gg/gosdk/internal/schemacheck/xsd.go 80.14% (+80.14%) 141 (+141) 113 (+113) 28 (+28) 🌟

Please note that the "Total", "Covered", and "Missed" counts above refer to code statements instead of lines of code. The value in brackets refers to the test coverage of that file in the old version of the code.

Changed unit test files

  • github.com/oddin-gg/gosdk/internal/schemacheck/internal_test.go
  • github.com/oddin-gg/gosdk/internal/schemacheck/schemacheck_test.go
  • github.com/oddin-gg/gosdk/internal/schemacheck/source_test.go

@dsaiko
dsaiko merged commit db6019e into main Sep 14, 2026
7 checks passed
@dsaiko
dsaiko deleted the test/xsd-conformance branch September 14, 2026 14:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants