|
| 1 | +# VSC v2.8.1 — Comparison Runner Script |
| 2 | + |
| 3 | +Release: v2.8.1 |
| 4 | +Status: released |
| 5 | +Project: DigiEmu VSC / VSC Core |
| 6 | + |
| 7 | +## Summary |
| 8 | + |
| 9 | +VSC v2.8.1 adds the first executable comparison runner for the Node/Go conformance comparison profile introduced in v2.8. |
| 10 | + |
| 11 | +The runner reads the v2.7 conformance fixture index, executes the Go verifier against each fixture, parses the machine-readable JSON result, and compares the actual verification class against the expected fixture result. |
| 12 | + |
| 13 | +Core principle: |
| 14 | + |
| 15 | +```text |
| 16 | +same fixture input → same verification result class |
| 17 | +``` |
| 18 | + |
| 19 | +## Added |
| 20 | + |
| 21 | +- New comparison runner script: |
| 22 | + |
| 23 | +```text |
| 24 | +scripts/compareConformanceResults.js |
| 25 | +``` |
| 26 | + |
| 27 | +- New CLI command: |
| 28 | + |
| 29 | +```powershell |
| 30 | +npm run vsc -- compare:fixtures |
| 31 | +``` |
| 32 | + |
| 33 | +## Updated |
| 34 | + |
| 35 | +- README documentation |
| 36 | +- v2.8 conformance documentation |
| 37 | +- VSC CLI command routing |
| 38 | + |
| 39 | +## Comparison behavior |
| 40 | + |
| 41 | +For each fixture, the runner: |
| 42 | + |
| 43 | +1. Reads `conformance/v2.7/fixture-index.json` |
| 44 | +2. Runs: |
| 45 | + |
| 46 | +```powershell |
| 47 | +go run ./cmd/vsc-go verify-bundle --json <fixture> |
| 48 | +``` |
| 49 | + |
| 50 | +3. Parses the JSON verification result |
| 51 | +4. Compares `actual result` against `expected_result` |
| 52 | +5. Derives the semantic exit-code class from the JSON `result` |
| 53 | +6. Prints a comparison table |
| 54 | +7. Returns final comparison status |
| 55 | + |
| 56 | +## Important Windows note |
| 57 | + |
| 58 | +On Windows, `go run` may flatten or obscure non-zero exit codes in some execution contexts. |
| 59 | + |
| 60 | +For this reason, the comparison runner treats the JSON field: |
| 61 | + |
| 62 | +```json |
| 63 | +"result": "PASS" | "FAIL" | "ERROR" |
| 64 | +``` |
| 65 | + |
| 66 | +as the primary semantic truth and derives the expected exit-code class from it. |
| 67 | + |
| 68 | +## Validated fixtures |
| 69 | + |
| 70 | +```text |
| 71 | +pass-basic → COMPARE_PASS |
| 72 | +fail-checksum-mismatch → COMPARE_PASS |
| 73 | +error-malformed-manifest → COMPARE_PASS |
| 74 | +``` |
| 75 | + |
| 76 | +Expected final output: |
| 77 | + |
| 78 | +```text |
| 79 | +Final result: COMPARE_PASS |
| 80 | +``` |
| 81 | + |
| 82 | +## Validation commands |
| 83 | + |
| 84 | +```powershell |
| 85 | +npm run vsc -- compare:fixtures |
| 86 | +npm run vsc -- verify-all |
| 87 | +``` |
| 88 | + |
| 89 | +Expected: |
| 90 | + |
| 91 | +```text |
| 92 | +compare:fixtures → Final result: COMPARE_PASS |
| 93 | +verify-all → FAIL: 0 |
| 94 | +``` |
| 95 | + |
| 96 | +## Scope |
| 97 | + |
| 98 | +v2.8.1 does not change the verification semantics of VSC. |
| 99 | + |
| 100 | +It adds an executable comparison layer for checking that fixture-level conformance results remain aligned with the declared comparison profile. |
| 101 | + |
| 102 | +## Known cosmetic issue |
| 103 | + |
| 104 | +The comparison table header is currently functional but visually tight: |
| 105 | + |
| 106 | +```text |
| 107 | +fixture_id expectedactual exp_exitact_exitcomparison |
| 108 | +``` |
| 109 | + |
| 110 | +This can be improved in a follow-up patch, for example: |
| 111 | + |
| 112 | +```text |
| 113 | +v2.8.2 — Runner table formatting + optional --json output |
| 114 | +``` |
| 115 | + |
| 116 | +## Release meaning |
| 117 | + |
| 118 | +v2.8.1 turns the v2.8 Node/Go comparison draft from documentation into an executable conformance check. |
| 119 | + |
| 120 | +This strengthens VSC as a portable evidence verification layer by making verifier behavior more reproducible, comparable, and testable across implementations. |
0 commit comments