refactor: separate application-specific wasm tooling

.cargo/config.toml

@@ -0,0 +1,2 @@
+[alias]
+xt = "run --quiet --release -p unwasm-xtask --"

.github/workflows/ci.yml

@@ -96,8 +96,8 @@ jobs:
 
   # The tier that needs the real modules: ~19 MB of captures to download, so it
   # does not run on every push. Start it by hand from the Actions tab, or let
-  # the weekly run take it — which also catches WhatsApp rolling a capture out
-  # of the published archive, since that is what the fetch script fails on.
+  # the weekly run take it. whatspec's wa-store restores each file by its
+  # pinned content identity, independent of the current live rollout.
   captures:
     if: github.event_name != 'push' && github.event_name != 'pull_request'
     runs-on: ubuntu-latest
@@ -126,7 +126,7 @@ jobs:
           echo "$HOME/emsdk/upstream/emscripten" >> "$GITHUB_PATH"
 
       - name: Fetch the captures
-        run: ./scripts/fetch-captures.sh
+        run: cargo xt fetch-captures
 
       # The whole tier, including the tests that hand generated Rust back to
       # rustc. It used to be thirty-one minutes and only the decompilation was
@@ -136,7 +136,7 @@ jobs:
       - name: Every capture, decompiled — and one of them run
         run: cargo test --test captured -- --ignored --nocapture
 
-      - name: The VoIP module's host, names, registrations and trampolines
+      - name: "Large Emscripten corpus: host, names, registrations and trampolines"
         run: |
           cargo test --test host --test names --test embind --test trampolines \
             -- --ignored --nocapture

.gitignore

@@ -1,6 +1,10 @@
 /target
+/.cache/
 /fixtures/wasm
 # The captured modules. WhatsApp's build output, fetched by
 # scripts/fetch-wasm.py and verified against wasm.lock.json; a copy committed
 # here would drift from the capture every recorded offset was read out of.
 /wasm
+
+/.derive-mlow*/
+**/__pycache__/

AGENTS.md

@@ -3,32 +3,20 @@
 A WebAssembly decompiler that emits Rust. Read `README.md` first — it holds the
 level model, what the real modules do, and the known limits.
 
-## This workspace holds two projects, and they are deliberately not one
-
-`crates/oracle-core` and `crates/oracle-cli` are the other half: they run the
-original `.wasm` under wasmtime instead of decompiling it. **`AGENTS-oracle.md`
-is their guide and it is not optional reading before touching them** — it
-carries the host-environment rules, the capture-lock rules, and a table of
-hypotheses already ruled out by measurement.
-
-Three things that catch people out when working across both:
-
-- **The duplication between the two host environments is the design.** See the
-  long note in `Cargo.toml`. If you are about to remove it, read RFC-0005 of
-  `wa-codegen-research` first.
-- **`oracle-core` may depend on `unwasm-core`; never the reverse.** The
-  decompiler's one dependency is a property its output relies on. Today the
-  dependency is used in exactly one place, `carry.rs`.
-- **Lints differ per crate in exactly one respect.** The decompiler keeps
-  `unsafe_code = "forbid"`; the oracle sets `deny`, because reading wasmtime's
-  shared memory needs `unsafe` and each site carries a SAFETY note. Everything
-  else, `missing_docs` included, is the workspace's and is enforced on both
-  halves — the 154 undocumented public items the oracle arrived with are paid
-  off, so `-D warnings` is clean across the workspace and stays that way.
-
-Running `cargo test --workspace` runs both, which takes about four minutes
-because the oracle brings up a PJSIP worker pool. `cargo test -p unwasm-core`
-is still the fast loop.
+## Repository boundary
+
+Only application-agnostic decompilation belongs here. A downstream executable
+oracle may depend on `unwasm-core` for static facts such as fingerprints, but
+`unwasm-core` must never depend on an application host, capture catalog,
+protocol crate or codec recipe. WhatsApp-specific execution and MLOW derivation
+live in `whatsapp-rust/tools`; verified capture transport lives in whatspec's
+`wa-store` crate.
+
+## Repository tasks
+
+Use `cargo xt --help` for the external regression corpus. It delegates verified
+capture restoration to a commit-pinned `wa-store`; no HTTP/archive machinery
+or application recipe belongs to this repository.
 
 ## Build & verify
 
@@ -568,7 +556,7 @@ easy to get wrong:
 ## The corpus rolls, and a measurement is dated by the build it was taken on
 
 WhatsApp reissues these payloads under new ids and stops serving the old ones.
-`scripts/fetch-captures.sh` pulls what is still published from the public
+`cargo xt fetch-captures` pulls what is still published from the public
 `oxidezap/whatspec` archive and checks each against a pinned sha256; the two
 captures more than one test names are constants in `tests/common/mod.rs`.
 
@@ -585,40 +573,19 @@ which build you measured.
 
 ## Open work
 
-1. **Drive the VoIP module.** `unwasm host` writes **103 methods for
-   `JgwtTQVeWPm`, 35 of them still `todo!()`**. Its allocator already runs:
-   `captured.rs` compiles a 42-function slice and compares it against the
-   engine, linear memory included. What the remaining 35 are is worth knowing
-   before adding to them:
-
-   - **22 are WhatsApp's own callbacks** — eleven `call_*_js_sync` capture and
-     playback drivers, plus `on_call_event_js_sync`, `renderVideoFrame_js`,
-     `sendSignalingXMPP_js_sync` and the rest. Nothing but the application can
-     answer them.
-   - **6 are the C++ catch-matching and primary-exception entry points**,
-     refused on purpose — see the note on `__cxa_find_matching_catch_*` above.
-   - **2 are `asm_const`**, JavaScript the module carries.
-   - the rest are `gethostbyname`, the offscreen canvas, `longjmp`,
-     `emscripten_receive_on_main_thread_js` and the mailbox postmessage.
-
-   The last two are the only ones a capability would unblock, and neither is a
-   missing capability — `emscripten_receive_on_main_thread_js` marshals its
-   arguments as *doubles* and relies on JavaScript coercing each one to the
-   callee's parameter type, and reproducing that coercion is a guess about a
-   conversion nobody here can check. It stays a `todo!()` for the same reason
-   `__cxa_find_matching_catch_*` does.
-
-   What is *not* left is anything a standard already decides, or anything that
-   needed the instance. Both of those were written.
-2. **Level 2 — the rest of it.** The half that is in is the half the module
+Application callbacks are deliberately outside this list. `unwasm host`
+declares unresolved imports, but implementing a protocol callback belongs to
+the consuming application's host adapter.
+
+1. **Level 2 — the rest of it.** The half that is in is the half the module
    writes down: class names and vtables from the C++ RTTI (`--level 2`,
    `unwasm classes`), and what embind's registrations declare. What is still
    open is the half that has to be inferred — structs from access patterns, and
    parameter roles read from how the code uses them, in the manner of
    `wa-wasm-oracle`'s `abi.rs`: dereferenced means pointer, and the access width
    says what it points at. That half *is* speculative, and it needs the same
    treatment level 1 got — opt-in, and saying at every site what it rests on.
-3. **Improve what a catalogue recognises.** `--stub-recognised` leaves the
+2. **Improve what a catalogue recognises.** `--stub-recognised` leaves the
    bodies out now, so the size of the cut is exactly the catalogue's recall:
    ~91% across builds of one toolchain and a handful across emscripten
    versions. The mechanism is not the limit; the fingerprint is.