Skip to content

Commit 043f877

Browse files
sirdeggenclaude
sirdeggen
and
claude
committed
feat: add vector schema, VECTOR-FORMAT.md, META.json, and conformance CI gate
Formalises Phase 1 of the MBGA plan: introduces the cross-language conformance corpus structure (JSON Schema, documentation, corpus metadata) and wires a conformance CI job that validates all vector files on every push/PR after the main build-and-test job succeeds. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
1 parent 3da8eac commit 043f877

4 files changed

Lines changed: 317 additions & 0 deletions

File tree

‎.github/workflows/ci.yml‎

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -41,3 +41,27 @@ jobs:
4141

4242
- name: Test all packages
4343
run: pnpm -r run test
44+
45+
conformance:
46+
name: Conformance Vectors
47+
runs-on: ubuntu-latest
48+
needs: [build-and-test]
49+
steps:
50+
- uses: actions/checkout@v4
51+
- uses: pnpm/action-setup@v4
52+
with:
53+
version: 9
54+
- uses: actions/setup-node@v4
55+
with:
56+
node-version: 20
57+
cache: pnpm
58+
- run: pnpm install --ignore-scripts
59+
- name: Validate conformance vectors
60+
run: cd conformance/runner && node src/runner.js --validate-only
61+
- name: Upload conformance report
62+
if: always()
63+
uses: actions/upload-artifact@v4
64+
with:
65+
name: conformance-report
66+
path: conformance/reports/
67+
if-no-files-found: ignore

‎conformance/META.json‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
{
2+
"version": "1.0.0",
3+
"description": "BSV cross-language conformance vector corpus",
4+
"domains": ["sdk", "wallet", "overlay", "messaging", "broadcast", "auth"],
5+
"brc_coverage": {
6+
"BRC-42": ["sdk.keys.key-derivation"],
7+
"BRC-31": [],
8+
"BRC-29": [],
9+
"BRC-100": []
10+
},
11+
"stats": {
12+
"total_files": 0,
13+
"total_vectors": 0,
14+
"last_updated": "2026-04-24"
15+
}
16+
}

‎conformance/VECTOR-FORMAT.md‎

Lines changed: 226 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,226 @@
1+
# BSV Conformance Vector Format
2+
3+
## Purpose
4+
5+
The conformance corpus is a language-neutral collection of test vectors that any BSV SDK implementation — TypeScript, Go, Rust, Python, etc. — must pass to be considered conformant. Each vector file captures a single behavioural domain (e.g. key derivation, ECDSA signing, BRC-29 message encryption) and provides concrete input/output pairs derived from the TypeScript reference implementation.
6+
7+
Goals:
8+
- Detect cross-language regressions before they reach production.
9+
- Give new SDK authors a single source of truth for expected behaviour.
10+
- Provide traceability from BRC specifications to executable test cases.
11+
12+
---
13+
14+
## File Structure
15+
16+
Vector files live under `conformance/vectors/` and are named after their stable ID, with dots replaced by slashes:
17+
18+
```
19+
conformance/
20+
META.json # corpus-level metadata
21+
schema/
22+
vector.schema.json # JSON Schema (2020-12) for every vector file
23+
vectors/
24+
sdk/
25+
keys/
26+
key-derivation.json # id: sdk.keys.key-derivation
27+
crypto/
28+
ecdsa.json # id: sdk.crypto.ecdsa
29+
wallet/
30+
brc100/
31+
interface.json
32+
...
33+
runner/
34+
src/
35+
runner.js # reference runner (Node.js)
36+
reports/ # CI output — gitignored
37+
```
38+
39+
### ID naming convention
40+
41+
The `id` field uses a stable dot-separated namespace that mirrors the directory path:
42+
43+
| Segment | Examples |
44+
|----------|-------------------------------|
45+
| domain | `sdk`, `wallet`, `overlay` |
46+
| area | `keys`, `crypto`, `brc100` |
47+
| feature | `key-derivation`, `ecdsa` |
48+
49+
Full example: `sdk.keys.key-derivation`
50+
51+
Segment rules:
52+
- Lowercase letters and digits only (`[a-z][a-z0-9]*`).
53+
- Hyphens are allowed within the *final* segment (e.g. `key-derivation`) but not between segments.
54+
- Once published, IDs are **permanent**. Rename = new ID + deprecate old.
55+
56+
### `parity_class` values
57+
58+
| Value | Meaning |
59+
|--------------|------------------------------------------------------------------|
60+
| `required` | All conformant implementations MUST pass these vectors. |
61+
| `intended` | Strong expectation; failure should be justified and tracked. |
62+
| `best-effort`| Optional; pass if the language/platform supports it. |
63+
| `unsupported`| Documented as out-of-scope for this domain (no vectors needed). |
64+
65+
### Tags
66+
67+
Tags are free-form strings attached to individual vectors for filtering:
68+
69+
| Tag example | Conventional meaning |
70+
|----------------|----------------------------------------------|
71+
| `happy-path` | Nominal, expected-success case |
72+
| `error-case` | Input that must produce a specific error |
73+
| `edge-case` | Boundary condition |
74+
| `brc-42` | Directly exercises a specific BRC |
75+
| `slow` | Long-running; may be excluded from fast CI |
76+
77+
---
78+
79+
## Vector Format
80+
81+
Each vector file is a JSON object that conforms to `conformance/schema/vector.schema.json`.
82+
83+
### Annotated example
84+
85+
```json
86+
{
87+
"$schema": "https://bsv-blockchain.github.io/ts-stack/conformance/schema/vector.schema.json",
88+
89+
// Stable ID — never changes after publication
90+
"id": "sdk.keys.key-derivation",
91+
92+
// Human-readable title
93+
"name": "BRC-42 HD Key Derivation",
94+
95+
// BRC specs this file exercises
96+
"brc": ["BRC-42"],
97+
98+
// Semantic version of THIS vector file (not the SDK)
99+
"version": "1.0.0",
100+
101+
// SDK version used to generate expected outputs
102+
"reference_impl": "ts-sdk@2.0.14",
103+
104+
// All conformant implementations must pass these
105+
"parity_class": "required",
106+
107+
"vectors": [
108+
{
109+
// Unique within this file; stable after publication
110+
"id": "derive-child-key-hardened",
111+
112+
"description": "Derive hardened child at index 0x80000000 from a known root key",
113+
114+
// All byte arrays encoded as lowercase hex strings
115+
"input": {
116+
"root_private_key_hex": "0102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f20",
117+
"path": "m/0'"
118+
},
119+
120+
"expected": {
121+
"child_private_key_hex": "...",
122+
"child_public_key_hex": "..."
123+
},
124+
125+
"tags": ["happy-path", "brc-42"],
126+
127+
// Optional: set true to skip in CI with a documented reason
128+
"skip": false
129+
}
130+
]
131+
}
132+
```
133+
134+
---
135+
136+
## Rules
137+
138+
1. **Stable IDs.** The `id` at file level and each vector `id` within a file are permanent once the file is merged to `main`. Changing them is a breaking change to all downstream runners.
139+
140+
2. **Hex encoding.** All binary data (keys, signatures, hashes, scripts) must be encoded as lowercase hexadecimal strings. Do not use base64, base58, or any other encoding.
141+
142+
3. **No secrets.** Test vectors are public. Never include real private keys, mnemonics, or credentials that control funds or production systems. Use synthetically generated keys.
143+
144+
4. **Append-only after publication.** Existing vectors must not be modified. If an expected value changes (e.g. a bug fix in the reference implementation), deprecate the old vector (`"skip": true, "skip_reason": "superseded by v2"`) and add a new vector with a new ID.
145+
146+
5. **Deterministic inputs.** If a function requires randomness (e.g. ECDSA nonce), use a fixed `nonce` field in `input` and document how the runner should inject it. Never rely on PRNG state.
147+
148+
6. **No implementation code in this directory.** The `conformance/` tree contains only data and the reference runner. SDK-specific test harnesses live in their own packages.
149+
150+
---
151+
152+
## How to Add a New Vector
153+
154+
1. Open the appropriate vector file (e.g. `conformance/vectors/sdk/crypto/ecdsa.json`).
155+
2. Append a new object to the `"vectors"` array.
156+
3. Choose an `id` that is unique within the file and descriptive (`sign-recoverable-low-s`, not `test3`).
157+
4. Run the reference TypeScript implementation to produce the `expected` values.
158+
5. Validate the file: `node conformance/runner/src/runner.js --validate-only`.
159+
6. Open a PR. CI will validate the schema and run all vectors.
160+
161+
---
162+
163+
## How to Add a New Vector File
164+
165+
1. Decide the stable ID (e.g. `sdk.crypto.schnorr`).
166+
2. Create the directory path: `conformance/vectors/sdk/crypto/`.
167+
3. Create `schnorr.json` following the annotated example above.
168+
4. Add the new ID to `conformance/META.json` under the appropriate `brc_coverage` entry.
169+
5. Update `stats.total_files` and `stats.last_updated` in `META.json`.
170+
6. Validate and open a PR.
171+
172+
---
173+
174+
## Runner Contract
175+
176+
The reference runner lives at `conformance/runner/src/runner.js` and is invoked by CI.
177+
178+
### CLI flags
179+
180+
| Flag | Effect |
181+
|-------------------|----------------------------------------------------------------|
182+
| `--validate-only` | Parse and schema-validate all vector files; do not execute. |
183+
| `--filter <glob>` | Run only vector files matching the glob (e.g. `sdk.keys.*`). |
184+
| `--report` | Write a JSON summary to `conformance/reports/report.json`. |
185+
| `--verbose` | Print per-vector pass/fail lines. |
186+
187+
### Exit codes
188+
189+
| Code | Meaning |
190+
|------|------------------------------------------------------|
191+
| `0` | All executed vectors passed (or validate-only clean). |
192+
| `1` | One or more vectors failed. |
193+
| `2` | Schema validation error or malformed vector file. |
194+
195+
### Report format (`conformance/reports/report.json`)
196+
197+
```json
198+
{
199+
"generated_at": "<ISO-8601>",
200+
"total": 42,
201+
"passed": 40,
202+
"failed": 1,
203+
"skipped": 1,
204+
"results": [
205+
{
206+
"file": "sdk/keys/key-derivation.json",
207+
"vector_id": "derive-child-key-hardened",
208+
"status": "passed"
209+
}
210+
]
211+
}
212+
```
213+
214+
---
215+
216+
## Cross-Language Runner Locations
217+
218+
Each SDK implementation provides its own runner that reads from this shared corpus:
219+
220+
| Repository | Runner path | Language |
221+
|-------------------|--------------------------------------------------|------------|
222+
| `ts-stack` | `conformance/runner/src/runner.js` | TypeScript |
223+
| `go-sdk` (planned)| `conformance/runner/runner.go` | Go |
224+
| `rust-sdk` (planned)| `conformance/runner/src/main.rs` | Rust |
225+
226+
All runners MUST implement the same CLI contract (flags and exit codes) defined above. The CI job in each repository pins the conformance corpus via a git submodule or a direct checkout of this repository at a tagged commit.

‎conformance/schema/vector.schema.json‎

Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,51 @@
1+
{
2+
"$schema": "https://json-schema.org/draft/2020-12/schema",
3+
"$id": "https://bsv-blockchain.github.io/ts-stack/conformance/schema/vector.schema.json",
4+
"title": "BSV Conformance Vector File",
5+
"description": "Schema for cross-language conformance test vector files",
6+
"type": "object",
7+
"required": ["id", "name", "version", "reference_impl", "parity_class", "vectors"],
8+
"properties": {
9+
"$schema": { "type": "string" },
10+
"id": {
11+
"type": "string",
12+
"pattern": "^[a-z][a-z0-9]*(?:\\.[a-z][a-z0-9]*)*$",
13+
"description": "Stable dot-separated identifier, e.g. sdk.crypto.ecdsa"
14+
},
15+
"name": { "type": "string" },
16+
"brc": {
17+
"type": "array",
18+
"items": { "type": "string" },
19+
"description": "Related BRC numbers, e.g. ['BRC-42']"
20+
},
21+
"version": {
22+
"type": "string",
23+
"pattern": "^\\d+\\.\\d+\\.\\d+$"
24+
},
25+
"reference_impl": { "type": "string", "description": "e.g. ts-sdk@2.0.14" },
26+
"parity_class": {
27+
"type": "string",
28+
"enum": ["required", "intended", "best-effort", "unsupported"]
29+
},
30+
"vectors": {
31+
"type": "array",
32+
"minItems": 1,
33+
"items": {
34+
"type": "object",
35+
"required": ["id", "description", "input", "expected"],
36+
"properties": {
37+
"id": { "type": "string" },
38+
"description": { "type": "string" },
39+
"input": { "type": "object" },
40+
"expected": { "type": "object" },
41+
"tags": {
42+
"type": "array",
43+
"items": { "type": "string" }
44+
},
45+
"skip": { "type": "boolean", "description": "If true, skip in CI" },
46+
"skip_reason": { "type": "string" }
47+
}
48+
}
49+
}
50+
}
51+
}

0 commit comments

Comments
 (0)