Compare the WASM and native builds of the same operation

[jorgeMFS]

Closes the harness half of E3 (biochef-agent#16).

The gap

A recipe declaring runtime.modes: [wasm, native] promises an operation means
the same thing whichever runtime runs it. BioChef depends on that in a way that
is easy to miss: the editor runs the WASM build in the browser through Aioli,
and the Agent runs the native build as a subprocess.

Nothing has ever checked they agree. hub.py test runs runtime/native/<bin>
and only that, so the artifact every browser user runs is the one nothing ran.

Seven of the thirty-one recipes declare both runtimes — fastp, gto, ksw2, lastz,
prodigal, samblaster, seqtk — spanning 84 operations.

What this adds

python hub/hub.py parity --registry-dir registry --report parity-report.json

One invocation per operation, run against both builds, comparing stdout, exit
status and declared file outputs. Both sides get byte-identical inputs and
identical argument strings, so a tool that echoes its own arguments cannot look
divergent because the harness passed a path to one side and a basename to the
other.

It runs the WASM artifact exactly as published — web,worker,
MODULARIZE=1, INVOKE_RUN=0 — by handing the module its wasmBinary
directly, so it never reaches the environment detection that would look for a
browser fetch(). Rebuilding it node-friendly would have been easier and would
have compared an artifact nobody runs.

Two outcomes kept deliberately apart

outcome	fails the run?
declared both runtimes, could not be compared	yes — the recipe’s promise was not kept
single-runtime recipe	no — never in scope

Conflating those is how this kind of gate rots into a suggestion. It is also why
there is no --min-compared in per-PR validation: a PR touching only
single-runtime recipes has nothing to compare and must not fail for it.

Both runtimes failing identically is parity, but no evidence the operation
works, so it is counted separately and never satisfies the floor.

Checking the checker

hub.py parity --self-test asks the harness to prove it still fails on a
divergence, and CI runs it immediately before the report. A comparison that
quietly stops comparing looks exactly like a catalogue that agrees.

End to end, I rebuilt ksw2’s WASM side with a different default match score:

  DIFFER  ksw2.align
           - stdout differs
           native: rc=0 stdout=(seq seq 196 196 97 97 98M)
           wasm  : rc=0 stdout=(seq seq 294 294 97 97 98M)
  FAIL: 1 operation(s) differ between wasm and native   exit=1

That exercise found a real bug in my own runner. callMain returns main’s
value rather than throwing when a program reports failure by returning, so
reading only the thrown ExitStatus recorded 0 for every such tool. ksw2 with no
arguments does exactly that, and it was reported as a divergence that was
entirely the harness’s fault. Fixed, and the comment says why.

What it does not cover, stated in the docs

One invocation per operation, from the required parameters. Anything behind
an optional parameter is not exercised — and for ksw2 that is the uncomfortable
part: it selects among seven scalar and SSE implementations through an optional
-t, and SSE is exactly where divergence is most plausible, since on x86 native
compiles -march=native against real SSE while WASM goes through emscripten’s
SSE-over-wasm-SIMD compatibility headers.

I swept that by hand — 385 invocations across all seven algorithms, four
lengths, four divergence rates, band-width and Z-drop variants, and seven
adversarial shapes (homopolymers, tandem repeats, Ns, one-long-one-short):
no divergence. A result for ksw2, not a general one, and CI does not repeat
it. Declaring cases per operation needs a schema key that does not exist yet
(#39).

Not fixed here

hub.py test prints failing tools and exits 0, and passes vacuously in three
other ways (#38). Making those fatal turns the pipeline red for whatever is
failing today — a decision with a blast radius that should not ride along inside
a PR adding a new check.

Caveat for review

I have no Docker locally, so recipes using the biowasm strategy could not be
built here; the four emscripten dual-mode recipes could. Everything above was
run against a real ksw2 bundle built both ways with the pinned emsdk 4.0.18. CI
is the first place this runs over the actual catalogue.

---

The hub now has CI on its own pull requests

It did not before. Both existing workflows are workflow_call and run only when
biochef-recipes invokes them, pinned at @master — so a change here is
exercised for the first time after it is merged, inside somebody else’s pull
request. That is an awkward position for the repository that decides what gets
published, and a worse one for a check whose whole job is to fail loudly.

The added workflow is narrow on purpose: the hub’s modules parse, the WASM
runner parses, and the parity harness proves it can still see a divergence. The
recipe build and tool tests need Docker, emscripten and a catalogue, and stay in
the reusable workflow where they already run.

Green on this PR:

PASS  Every hub module parses
PASS  The WASM runner parses
PASS  The parity harness can still see a divergence

What is still unverified, and cannot be verified before merge

The change to validate-recipes.yml — the steps that run parity over a real
catalogue — is not exercised by this PR, because that workflow only runs
when biochef-recipes calls it at @master. The first real run over built
bundles happens after merge, on the next recipes PR.

What I could verify locally I did: a real ksw2 bundle built both ways with the
pinned emsdk 4.0.18, driven through hub.py parity end to end, including a
planted divergence and the four ways the report could have been vacuous
(empty registry, all-skipped, out-of-scope-only, agreed-failure). Recipes using
the biowasm strategy need Docker, which I do not have here.

---

Audit: two more harness bugs, both of which would have cried wolf

ksw2 has neither a stdin input nor a file output, so the first version was green
while two of its paths had never run. I compiled a purpose-built probe — reads
stdin, reports the byte count, writes out.txt — to WASM and compared it
against its native build.

stdin never arrived. It was installed with FS.init() on the resolved
module, which throws ErrnoError because the filesystem is already initialised.
The throw is not the damaging part: the program still ran and read zero
bytes. Every stdin-driven tool in the catalogue — seqtk, jq, sed, grep, gawk —
would have been reported as diverging from its native build when all that
happened is the harness never fed it. Emscripten takes stdin from
Module[stdin] during startup, so it must be handed to the factory.

File outputs used two different matching rules. The WASM side read the
literal declared name while the native side searched by stem. A bundle
declaring out against a tool writing out.txt would have held a file on one
side and null on the other. Both sides now apply the same rule, and the native
side no longer offers the copied binary as a candidate, since the WASM
filesystem has no binary in it.

That is three harness bugs counting callMain — and all three produce false
divergences on tools that are fine. That is the failure mode that matters here:
a parity gate that cries wolf gets switched off, and a switched-off gate is
indistinguishable from the one nobody wrote.

So runner_check.mjs now pins both against a stub module encoding emscripten’s
observed contract. It needs no toolchain and runs in CI. Verified by mutation —
reverting either fix fails it:

[FAIL] stdin must reach the module, not be dropped on the floor
[FAIL] a declared output must be matched by stem, as the native side matches it

Also fixed: a missing node now says so instead of raising FileNotFoundError
from inside subprocess, and a corrupt bundle.json is reported as an
unreadable bundle that fails the run rather than taking the whole catalogue down
with it. Both were reachable; neither said anything useful.

Full regression, every scenario:

ksw2 both ways                     exit=0  OK
ksw2 with planted divergence       exit=1  1 operation(s) differ
probe: file output differs only    exit=1  1 operation(s) differ
declares both, native missing      exit=1  could not be compared
wasm-only recipe                   exit=0  OK (out of scope)
corrupt bundle.json                exit=1  could not be compared