Skip to content

feat(mlow): complete codec evidence and own oracle tooling - #1447

Merged
jlucaso1 merged 34 commits into
mainfrom
mlow-wasm-vectors
Sep 6, 2026
Merged

jlucaso1 merged 34 commits into
mainfrom
mlow-wasm-vectors

Conversation

@jlucaso1

@jlucaso1 jlucaso1 commented Sep 5, 2026 •

Copy link
Copy Markdown
Member

MLOW fixtures are reproducible against pinned WhatsApp Web J/S wasm captures. This PR completes the codec corpus, replaces repository Python/Bash automation with Rust cargo xt tasks, and keeps WhatsApp-specific oracle execution in unpublished host tooling.

  • unwasm-core supplies generic static analysis through a pinned Git dependency; WhatSpec wa-store restores verified captures. Neither enters the application runtime graph. Both producer PRs are merged.
  • cargo xt is a lightweight dispatcher for hashes, descriptors and CI tasks. Capture-backed commands run through the separate release whatsapp-oracle-task package. Bases, typed recipes and locked hashes replace eight expanded specs in Git; CI regenerates and uploads the exact expansions. This removes about 122,000 added lines without changing derivation hashes or fixture payloads.
  • MLOW coverage includes 60/120 ms encode/decode, DTX, VAD, front-end, pitch, LSF, range coding, excitation/noise and postfilters. Compact CBOR/zstd preserves canonical payloads and independent C auditors.
  • The oracle shares process-wide WASI files, descriptors and streams across workers, rejects unsupported file flags and incomplete embind signatures, propagates logging failures, and validates replacement bounds and rewritten wasm. Integer registration, ABI marshalling and pointer reads use one validated descriptor. Media diagnostics remain bounded; test helpers and experiment setup are shared.
  • WASI keeps open descriptors usable across unlink through one shared object per file, validates open directories, lookup flags, descriptor rights and import signatures, checks clock outputs before ticking, bounds sparse file growth, prestat buffers, path lengths, standard streams and argv writes, and caps snapshot records independently of bytes. Unlink also validates its descriptor and rejects non-UTF-8 paths, as does open. Legalized i64 call forms are decoded, not just accepted. Media trace comparison includes the callback symbol; adapters map both sides to one canonical boundary name first.
  • Guest workers and emval handles are capped per runtime with byte budgets; file-backed mmaps are refused; data-segment offsets require a single constant expression; string scans require a positive minimum; raw audit archives pin their exact shipped bytes in the manifest. Stack readiness waits on an elapsed-time deadline; the recorded table is the single funcref table; embind registrations require dispatchable operands; cleanup failures surface; tail calls are marked as calls and returns. Settle rechecks quiescence after the final drain; table elements decode constant ref.func expressions; exception destructors run at catch end; the offer guard is located through decoded operators in a validated module.
  • Operational contracts are separated from archived investigations. Guest threads can execute concurrently; cooperative scheduling is not a memory-safety guarantee. TASK.md is removed from version control.

cargo xt oracle conformance combines generated WhatSpec checks, capture-backed tests, J/S rederivation and Rust VoIP/IQ tests. Audio/video trace capture and comparison support future differential adapters; full callback ABIs and end-to-end IQ/signaling equivalence remain explicit coverage gaps. The normal suite ignores 26 slow signaling research scenarios; --slow selects them separately.

Local validation: oracle-core lib (68 tests), oracle-task/cli (14 tests) and fast capture-backed integration suites passed via nextest with no missing-capture skips; a real-engine boot validated the worker, mmap, rights and signature bounds. Clippy for all targets of the four tooling crates, formatting, and dependency checks for those crates passed. Heavy threading, rederivation, rustdoc and workspace gates run in CI. Merged main through #1448; the only merge conflict was the sqlite-storage Cargo.lock entries.

@greptile-apps

greptile-apps Bot commented Sep 5, 2026 •

Copy link
Copy Markdown

Too many files changed for review (196 files, 150 file limit).

Bypass the limit by tagging @greptile-apps to review.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 5, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-09-06T14:36:51.952545Z 0bf70ba New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@coderabbitai

coderabbitai Bot commented Sep 5, 2026 •

Copy link
Copy Markdown

Important

Review skipped

Too many files!

This PR contains 173 files, which is 23 over the limit of 150.

To get a review, reduce the PR to 150 files or fewer by splitting it into smaller PRs or changing its base branch.

Upgrade to a paid plan to raise the limit.

This review couldn't start because sufficient usage credits or metered capacity aren't available. Add credits or update usage-based reviews in the billing tab, then retry.

Check out review usage here.

⚙️ Run configuration

Configuration used: Repository UI (base), Organization UI (inherited)

Review profile: ASSERTIVE

Plan: Team

Run ID: 68965ec3-42dc-46e2-8576-29816b1c15ba

📥 Commits

Reviewing files that changed from the base of the PR and between dc6c9a3 and 910fc7c.

⛔ Files ignored due to path filters (23)
  • Cargo.lock is excluded by !**/*.lock
  • tools/oracle-core/tests/fixtures/tiny.jpg is excluded by !**/*.jpg
  • tools/oracle-core/tests/fixtures/tiny.mp4 is excluded by !**/*.mp4
  • wacore/src/voip/mlow/testdata/e2e_vectors.cbor.zst is excluded by !**/*.zst
  • wacore/src/voip/mlow/testdata/exc_pre_lags.cbor.zst is excluded by !**/*.zst
  • wacore/src/voip/mlow/testdata/fe_dump.cbor.zst is excluded by !**/*.zst
  • wacore/src/voip/mlow/testdata/gennoise_params_dump.cbor.zst is excluded by !**/*.zst
  • wacore/src/voip/mlow/testdata/gennoise_vectors.cbor.zst is excluded by !**/*.zst
  • wacore/src/voip/mlow/testdata/harm_postfilter_vectors.raw.zst is excluded by !**/*.zst
  • wacore/src/voip/mlow/testdata/hp_postfilter_vectors.raw.zst is excluded by !**/*.zst
  • wacore/src/voip/mlow/testdata/lsf_quant_io.cbor.zst is excluded by !**/*.zst
  • wacore/src/voip/mlow/testdata/pitchio_ground_truth.cbor.zst is excluded by !**/*.zst
  • wacore/src/voip/mlow/testdata/pulse_vectors.cbor.zst is excluded by !**/*.zst
  • wacore/src/voip/mlow/testdata/rc_vectors.cbor.zst is excluded by !**/*.zst
  • wacore/src/voip/mlow/testdata/sigmode_ground_truth.cbor.zst is excluded by !**/*.zst
  • wacore/src/voip/mlow/testdata/wasm_fe.cbor.zst is excluded by !**/*.zst
  • wacore/src/voip/mlow/testdata/wasm_gennoise.cbor.zst is excluded by !**/*.zst
  • wacore/src/voip/mlow/testdata/wasm_harm_postfilter.cbor.zst is excluded by !**/*.zst
  • wacore/src/voip/mlow/testdata/wasm_hp_postfilter.cbor.zst is excluded by !**/*.zst
  • wacore/src/voip/mlow/testdata/wasm_lsf_quant.cbor.zst is excluded by !**/*.zst
  • wacore/src/voip/mlow/testdata/wasm_params.cbor.zst is excluded by !**/*.zst
  • wacore/src/voip/mlow/testdata/wasm_pitch.cbor.zst is excluded by !**/*.zst
  • wacore/src/voip/mlow/testdata/wasm_signal_mode.cbor.zst is excluded by !**/*.zst
📒 Files selected for processing (173)
  • .cargo/config.toml
  • .github/workflows/binary-size.yml
  • .github/workflows/codspeed.yml
  • .github/workflows/copilot-setup-steps.yml
  • .github/workflows/e2e.yml
  • .github/workflows/main.yml
  • .github/workflows/mlow-derive.yml
  • .github/workflows/release.yml
  • .github/workflows/supply-chain.yml
  • .github/workflows/wasm.yml
  • .gitignore
  • AGENTS.md
  • Cargo.toml
  • agent_docs/binary_size_ci.md
  • agent_docs/mlow_derivation.md
  • agent_docs/voip_audio_codecs.md
  • agent_docs/voip_conformance.md
  • agent_docs/voip_media_oracle.md
  • agent_docs/voip_oracle_history.md
  • agent_docs/voip_oracle_status.md
  • deny.toml
  • scripts/ci/binary_size_report.py
  • scripts/ci/feature_matrix_crates.sh
  • scripts/ci/measure_binary_size.py
  • scripts/ci/nextest_timed.sh
  • scripts/ci/sync-bartender-image.sh
  • scripts/ci/test_features.sh
  • scripts/regenerate-mlow-vectors.sh
  • scripts/regenerate-proto-desc.sh
  • scripts/regenerate-tables-desc.sh
  • scripts/regenerate-wire-desc.sh
  • storages/sqlite-storage/build.rs
  • tools/oracle-cli/Cargo.toml
  • tools/oracle-cli/src/main.rs
  • tools/oracle-core/AGENTS.md
  • tools/oracle-core/Cargo.toml
  • tools/oracle-core/README.md
  • tools/oracle-core/examples/call_the_sender.rs
  • tools/oracle-core/examples/common/mod.rs
  • tools/oracle-core/examples/free_trap.rs
  • tools/oracle-core/examples/init_stress.rs
  • tools/oracle-core/examples/mozjpeg_call.rs
  • tools/oracle-core/examples/offer_differential.rs
  • tools/oracle-core/examples/offer_probe.rs
  • tools/oracle-core/examples/outbound_after_settings.rs
  • tools/oracle-core/examples/outbound_setup_matrix.rs
  • tools/oracle-core/examples/outgoing_call.rs
  • tools/oracle-core/examples/profiler_flag.rs
  • tools/oracle-core/examples/ring_corruption.rs
  • tools/oracle-core/examples/self_participant_probe.rs
  • tools/oracle-core/examples/settings_probe.rs
  • tools/oracle-core/examples/signaling_census.rs
  • tools/oracle-core/examples/table_slot_of.rs
  • tools/oracle-core/examples/voip_flow.rs
  • tools/oracle-core/examples/voprf_flow.rs
  • tools/oracle-core/specs/frame0.raw
  • tools/oracle-core/specs/frames3.raw
  • tools/oracle-core/specs/mlow.lock.json
  • tools/oracle-core/specs/mlow_110frames.json
  • tools/oracle-core/specs/mlow_120ms.json
  • tools/oracle-core/specs/mlow_3frames.json
  • tools/oracle-core/specs/mlow_dtx_off.json
  • tools/oracle-core/specs/mlow_encode.json
  • tools/oracle-core/specs/mlow_encode_s.json
  • tools/oracle-core/specs/mlow_factory_probe.json
  • tools/oracle-core/specs/mlow_open.json
  • tools/oracle-core/specs/mlow_probe.json
  • tools/oracle-core/specs/synth120_head.raw
  • tools/oracle-core/specs/synth_mic.raw
  • tools/oracle-core/src/abi.rs
  • tools/oracle-core/src/call.rs
  • tools/oracle-core/src/carry.rs
  • tools/oracle-core/src/catalog.rs
  • tools/oracle-core/src/cxa.rs
  • tools/oracle-core/src/data.rs
  • tools/oracle-core/src/derive.rs
  • tools/oracle-core/src/embind.rs
  • tools/oracle-core/src/emscripten.rs
  • tools/oracle-core/src/emval.rs
  • tools/oracle-core/src/exports.rs
  • tools/oracle-core/src/host.rs
  • tools/oracle-core/src/inspect.rs
  • tools/oracle-core/src/integer.rs
  • tools/oracle-core/src/lib.rs
  • tools/oracle-core/src/media_probe.rs
  • tools/oracle-core/src/migrate.rs
  • tools/oracle-core/src/patch.rs
  • tools/oracle-core/src/runtime.rs
  • tools/oracle-core/src/schedule.rs
  • tools/oracle-core/src/shared.rs
  • tools/oracle-core/src/snapshot.rs
  • tools/oracle-core/src/state.rs
  • tools/oracle-core/src/threads.rs
  • tools/oracle-core/src/wasi.rs
  • tools/oracle-core/tests/abi_inference.rs
  • tools/oracle-core/tests/captured_modules.rs
  • tools/oracle-core/tests/carry.rs
  • tools/oracle-core/tests/common/mod.rs
  • tools/oracle-core/tests/differential.rs
  • tools/oracle-core/tests/embind_layouts.rs
  • tools/oracle-core/tests/fixtures/tiny.aac
  • tools/oracle-core/tests/fixtures/tiny.h264
  • tools/oracle-core/tests/host_environment.rs
  • tools/oracle-core/tests/media_probe.rs
  • tools/oracle-core/tests/media_tools.rs
  • tools/oracle-core/tests/patch.rs
  • tools/oracle-core/tests/signaling.rs
  • tools/oracle-core/tests/threading.rs
  • tools/oracle-core/tests/voip_oracle.rs
  • tools/oracle-core/wasm.lock.json
  • tools/oracle-task/Cargo.toml
  • tools/oracle-task/src/conformance.rs
  • tools/oracle-task/src/derive_mlow.rs
  • tools/oracle-task/src/derive_mlow/assemble.rs
  • tools/oracle-task/src/main.rs
  • tools/oracle-task/src/media.rs
  • tools/oracle-task/src/mlow-recipes.json
  • tools/oracle-task/src/mlow.rs
  • tools/oracle-task/src/oracle.rs
  • tools/oracle-task/src/oracle_patch.rs
  • tools/whatspec-codegen/Cargo.toml
  • tools/whatspec-codegen/src/main.rs
  • tools/xtask-support/Cargo.toml
  • tools/xtask-support/src/cbor.rs
  • tools/xtask-support/src/lib.rs
  • tools/xtask/Cargo.toml
  • tools/xtask/README.md
  • tools/xtask/src/ci.rs
  • tools/xtask/src/main.rs
  • tools/xtask/src/size.rs
  • tools/xtask/src/workflow.rs
  • wacore/Cargo.toml
  • wacore/build.rs
  • wacore/src/voip/mlow/decoder.rs
  • wacore/src/voip/mlow/fixture.rs
  • wacore/src/voip/mlow/golden.rs
  • wacore/src/voip/mlow/mod.rs
  • wacore/src/voip/mlow/param_decode_match.rs
  • wacore/src/voip/mlow/quality_tests.rs
  • wacore/src/voip/mlow/rangecoder.rs
  • wacore/src/voip/mlow/smpl_celpdec.rs
  • wacore/src/voip/mlow/smpl_gennoise.rs
  • wacore/src/voip/mlow/smpl_harm_postfilter.rs
  • wacore/src/voip/mlow/smpl_harmcomb.rs
  • wacore/src/voip/mlow/smpl_lpc.rs
  • wacore/src/voip/mlow/smpl_lsf_quant.rs
  • wacore/src/voip/mlow/smpl_pitch_enc.rs
  • wacore/src/voip/mlow/smpl_pulse.rs
  • wacore/src/voip/mlow/smpl_signal_mode.rs
  • wacore/src/voip/mlow/smpl_vad.rs
  • wacore/src/voip/mlow/testdata/PROVENANCE.md
  • wacore/src/voip/mlow/testdata/e2e_vectors.json
  • wacore/src/voip/mlow/testdata/exc_pre_lags.json
  • wacore/src/voip/mlow/testdata/fe_dump.json
  • wacore/src/voip/mlow/testdata/gennoise_params_dump.json
  • wacore/src/voip/mlow/testdata/gennoise_vectors.json
  • wacore/src/voip/mlow/testdata/harm_postfilter_vectors.raw
  • wacore/src/voip/mlow/testdata/hp_postfilter_vectors.raw
  • wacore/src/voip/mlow/testdata/lsf_quant_io.json
  • wacore/src/voip/mlow/testdata/packed-fixtures.json
  • wacore/src/voip/mlow/testdata/pitchio_ground_truth.json
  • wacore/src/voip/mlow/testdata/pulse_vectors.json
  • wacore/src/voip/mlow/testdata/rc_vectors.json
  • wacore/src/voip/mlow/testdata/sigmode_ground_truth.json
  • wacore/src/voip/mlow/testdata/wasm-fixtures.json
  • wacore/src/voip/mlow/testdata/wasm_derived_120ms_frames.json
  • wacore/src/voip/mlow/testdata/wasm_derived_120ms_ref.raw
  • wacore/src/voip/mlow/testdata/wasm_derived_frames.json
  • wacore/src/voip/mlow/testdata/wasm_derived_ref.raw
  • wacore/src/voip/mlow/testdata/wasm_derived_vad.json
  • wacore/tests/noise_handshake_test.rs
  • waproto/build.rs
  • waproto/src/lib.rs

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 58ce8101c7

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread .github/workflows/main.yml Outdated
Comment thread .github/workflows/mlow-derive.yml Outdated
Comment thread tools/oracle-core/tests/differential.rs
Comment thread tools/oracle-core/src/threads.rs Outdated
Comment thread tools/oracle-core/src/host.rs Outdated
Comment thread tools/oracle-core/src/catalog.rs Outdated
Comment thread tools/oracle-core/src/wasi.rs Outdated
Comment thread tools/oracle-core/src/runtime.rs
Comment thread tools/oracle-core/src/call.rs Outdated
Comment thread tools/oracle-core/src/state.rs Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 57c10e3b4a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread tools/oracle-core/tests/differential.rs Outdated
Comment thread tools/oracle-core/src/threads.rs Outdated
Comment thread tools/oracle-core/src/patch.rs
Comment thread tools/oracle-core/src/emscripten.rs Outdated
Comment thread tools/xtask/src/mlow.rs Outdated
@codspeed

codspeed Bot commented Sep 5, 2026 •

Copy link
Copy Markdown
Contributor

Merging this PR will improve performance by 10.58%

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 4 improved benchmarks
✅ 442 untouched benchmarks
⏩ 356 skipped benchmarks1

Performance Changes

Mode Benchmark BASE HEAD Efficiency
⚡ Memory crypt_payload_one 86 B 77 B +11.69%
⚡ Memory audio_unprotect_packet 86 B 77 B +11.69%
⚡ Memory sframe_decrypt 102 B 93 B +9.68%
⚡ Memory audio_protect_packet 106 B 97 B +9.28%

Tip

Curious why performance improved? Comment @codspeedbot explain why performance improved on this PR, or directly use the CodSpeed MCP with your agent.


Comparing mlow-wasm-vectors (910fc7c) with main (dc6c9a3)

Open in CodSpeed

Footnotes

  1. 356 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

@github-actions

github-actions Bot commented Sep 5, 2026 •

Copy link
Copy Markdown

📦 Binary size report

Metric main PR Δ
bin size (stripped) 10.51 MiB 10.50 MiB -6.69 KiB (-0.06%) 🔽
bin .text 8.47 MiB 8.46 MiB -7.81 KiB (-0.09%) 🔽
bin allocated (text+data+bss) 10.51 MiB 10.50 MiB -7.59 KiB (-0.07%) 🔽
llvm-lines wacore 574,629 574,629 0
llvm-lines wacore copies 18,840 18,840 0
llvm-lines whatsapp-rust lib 834,616 834,616 0
llvm-lines whatsapp-rust lib copies 26,741 26,741 0
deps crates (Cargo.lock) 470 604 +134 (+28.51%) ⚠️
.text per crate
Crate main PR Δ
.text whatsapp_rust 1.97 MiB 1.98 MiB +1.29 KiB (+0.06%) 🔺
.text wacore 788.21 KiB 787.88 KiB -334 B (-0.04%) 🔽
.text wacore_binary 84.13 KiB 84.14 KiB +8 B (+0.01%) 🔺
.text wacore_libsignal 191.19 KiB 191.04 KiB -145 B (-0.07%) 🔽
.text wacore_appstate 28.34 KiB 28.34 KiB 0
.text wacore_noise 22.21 KiB 22.21 KiB 0
.text waproto 1.74 MiB 1.74 MiB -37 B (-0.00%) 🔽
.text whatsapp_rust_sqlite_storage 604.35 KiB 604.78 KiB +444 B (+0.07%) 🔺
.text whatsapp_rust_tokio_transport 40.52 KiB 40.88 KiB +370 B (+0.89%) 🔺
.text whatsapp_rust_ureq_http_client 12.68 KiB 12.75 KiB +75 B (+0.58%) 🔺
.text std 1.03 MiB 1.02 MiB -9.56 KiB (-0.91%) 🔽
.text other deps 1.96 MiB 1.96 MiB +104 B (+0.01%) 🔺
Top movers (cargo-bloat attribution)
Crate main PR Δ
std 1.03 MiB 1.02 MiB -9.56 KiB (-0.91%)
metrics_exporter_prometheus 4.96 KiB (removed) -4.96 KiB (-100.00%)
rustix 191 B 2.45 KiB +2.26 KiB (+1214.14%)
ureq 92.64 KiB 94.14 KiB +1.51 KiB (+1.63%)
whatsapp_rust 1.97 MiB 1.98 MiB +1.29 KiB (+0.06%)

Baseline: dc6c9a340 (latest main run) · Head: 1a55bc0b5 · Graphs

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4b2ed410a3

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread tools/oracle-core/src/emscripten.rs Outdated
Comment thread tools/oracle-core/src/wasi.rs Outdated
Comment thread tools/oracle-core/src/call.rs Outdated
Comment thread tools/oracle-core/src/inspect.rs Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 8b8688d2d6

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread tools/oracle-core/tests/voip_oracle.rs Outdated
Comment thread tools/oracle-core/tests/signaling.rs
Comment thread tools/oracle-core/src/wasi.rs Outdated
Comment thread tools/oracle-core/src/runtime.rs Outdated
Comment thread tools/oracle-core/src/patch.rs
Comment thread tools/oracle-core/src/emscripten.rs Outdated
@jlucaso1

jlucaso1 commented Sep 5, 2026

Copy link
Copy Markdown
Member Author

@codspeedbot fix all regressions and noisy benchs

@codspeed

codspeed Bot commented Sep 5, 2026 •

Copy link
Copy Markdown
Contributor

@codspeedbot fix all regressions and noisy benchs

Pushed 3b4f34a6 (rebased onto the two oracle commits that landed meanwhile).

pitch_search (-13.21%) — real, and now paid back

Not noise. The corrected J#10736 weights (PITCH_DELTAWGHT 0.1439 → 0.3, PREVWGHT 0.7981 → 0.7) change which block tracks survive selection, so the full-resolution C/E update runs over more unique blocks. Confirmed two ways:

  • CI instruction time alone went 192.9 µs → 239.1 µs, so it is executed work, not the cross-runner cache-geometry effect the report warns about (that warning is real too — the two runs landed on different runners, which is why the headline reads +15% while a same-machine A/B reads +7%).
  • Reverting only those two constants locally: 311.8 µs → 291.2 µs.

The extra work is the point of the fix, so I bought it back inside smpl_pitch instead of touching the search. The flamegraph showed 28% of the row was calloc → memset: ~110 KB of scratch allocated and zeroed per internal frame, three times per 60 ms frame in production.

  • Dropped the coarse C/E/H snapshots — written every call, never read again (three zeroed allocations plus three full copies for nothing).
  • Stage-1 C is written straight into the head of the C array the upsample stages expand from, instead of a second buffer that was copied in.
  • Both E1 blocks are now filled as they are produced; every element was already written before being read, so vec![0.0; n] was paying for zeroing that was immediately overwritten.
  • The fine-search argmax memo is a dense array indexed by the (start subframe, segment length, block) triple the blockseg table bounds, replacing a HashMap whose hashing cost more than the argmax it saved — and whose RandomState made the row's cost vary per process, a documented source of run-to-run movement in this repo.

Same machine, codspeed run --mode simulation: 311.8 µs → 277.9 µs, i.e. below the 291.2 µs the row measures with the pre-J#10736 weights. Instruction time 227.8 µs → 200.0 µs. analyze_frame and mlow_encode* share the routine and can only benefit.

Bit-exactness: all 131 MLOW tests pass, including pitch_estimator_matches_c_ground_truth and pitch_estimator_matches_shipped_wasm (330 vectors), plus the 680 voip:: tests in a debug build so the tightened buffer bounds are checked. cargo fmt and clippy --all-targets -D warnings are clean.

bench_jid_push_phash_form — measured nothing

This row reported "no execution time" on both base and head: observing buf.as_bytes() after the call hands LLVM the pointer and the length but never the bytes, so it proved the formatting stores dead — exactly what the existing comment there was trying to prevent. The destination is now laundered through black_box before the write, which makes the pointer escape and forces the stores. The row measures 666 ns, in line with its siblings (to_non_ad_string 1.08 µs, to_string 1.80 µs).

Heads-up: on the next report this row moves from "< 1 ns" to a real number. That is the measurement starting to work, not a regression.

The rest of the report

  • bench_jid_parse[…@g.us] (+10.23%) and the ⚡ Memory rows are improvements, and the jid_parse family is ~85% simulated memory model on this hardware, so it moves whenever base and head land on different runner CPUs — the "different runtime environments" banner applies to it directly. Nothing to fix.
  • One row I deliberately left alone: identity_probe_misses[8] in signal_address_probe_benchmark.rs is a known-flaky row for the same RandomState reason, but it did not move here and de-noising it means changing what the benchmark models. Happy to do it in its own PR if you want.
Summary
  • Warming up the performance wizard...
  • Triage the reported regression and noisy rows — pitch_search is a real +15% (instruction time 192.9 -> 239.1 us): the corrected PITCH_DELTAWGHT/PREVWGHT keep a different survivor set, so more full-resolution work runs.
  • Reproduce pitch_search locally — Local Simulation A/B on one machine: 291.2 us with the old constants vs 311.8 us with the new ones. Flamegraph showed 28% of the row is calloc zeroing of per-call scratch.
  • Recover the pitch_search cost — 311.8 -> 277.9 us locally (-10.9%), below the 291.2 us pre-constant-change baseline; instruction time 227.8 -> 200.0 us. All 131 MLOW tests pass bit-exactly.
  • Fix the optimized-out benchmark row — bench_jid_push_phash_form measured nothing because only the buffer pointer/len was observed; laundering the destination through black_box first makes it measure 666 ns.
  • Validate benchmarks and tests — cargo fmt, clippy -D warnings on wacore + wacore-binary --all-targets, 680 voip tests in debug and the full wacore suite in release, plus full voip and jid benchmark suites under codspeed run --mode simulation.
  • Commit and reply — Pushed 3b4f34a onto mlow-wasm-vectors (rebased on the two oracle commits that landed meanwhile).
  • Handling comment — Comment handling finished successfully

Was this response helpful? React with 👍 or 👎 to help us improve CodSpeedBot.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a84c811004

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread tools/oracle-core/src/media_probe.rs
Comment thread tools/oracle-core/src/wasi.rs Outdated
Comment thread tools/oracle-core/src/wasi.rs Outdated
Comment thread tools/oracle-core/src/derive.rs
Comment thread tools/oracle-core/src/runtime.rs Outdated
Comment thread tools/oracle-core/src/patch.rs
codspeedbot and others added 2 commits September 5, 2026 23:13
The corrected J#10736 search weights keep a different set of survivor
tracks, which widens the full-resolution C/E update and costs ~10% more
work in `smpl_pitch` (CodSpeed read it as a 15% regression on
`codec_stages::pitch_search`). The extra work is the point of the fix, so
pay for it elsewhere in the same routine, all of it verified against the
existing goldens and the wasm oracle vectors:

- Drop the coarse C/E/H snapshots. They were written every call and never
  read again: three zeroed allocations and three full copies for nothing.
- Write stage-1 C straight into the head of the C array the upsample
  stages expand from, instead of into a second buffer that was copied in.
- Build both E1 blocks by filling as they are produced. Every element is
  written before it is read, so the zeroing a `vec![0.0; n]` pays for was
  pure overhead.
- Memoize the fine-search argmax in a dense array indexed by the
  (start subframe, segment length, block) triple the blockseg table
  bounds, rather than a `HashMap`. The hashing cost more than the argmax
  it saved, and its `RandomState` made the row's cost vary per process.

Locally under `codspeed run --mode simulation`, on one machine:
311.8 us before -> 277.9 us, i.e. below the 291.2 us the row measures with
the pre-J#10736 weights; instruction time 227.8 us -> 200.0 us. All 131
MLOW tests still pass, bit-exact.

`bench_jid_push_phash_form` measured no execution time at all, on this
branch and on main: observing `buf.as_bytes()` after the call hands LLVM
the pointer and the length but never the bytes, so the formatting stores
were dead. Laundering the destination through `black_box` before the
write makes the pointer escape; the row now measures 666 ns.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 041bfdc4d7

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread tools/oracle-core/src/runtime.rs Outdated
Comment thread tools/oracle-core/src/wasi.rs
Comment thread tools/oracle-core/src/abi.rs Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a47916d47e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread tools/oracle-cli/src/main.rs Outdated
Comment thread tools/oracle-core/src/patch.rs
Comment thread tools/oracle-core/src/wasi.rs Outdated
Comment thread tools/oracle-core/src/embind.rs Outdated
Comment thread tools/oracle-core/src/threads.rs

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 9833229f65

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread tools/oracle-core/src/runtime.rs Outdated
Comment thread tools/oracle-task/src/mlow.rs Outdated
Comment thread tools/oracle-core/src/derive.rs
Comment thread tools/oracle-core/examples/table_slot_of.rs Outdated
Comment thread tools/oracle-core/src/abi.rs
Comment thread tools/oracle-core/src/wasi.rs

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d5789a33fa

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread tools/oracle-core/src/wasi.rs
Comment thread tools/oracle-core/src/inspect.rs Outdated
Comment thread tools/oracle-core/src/call.rs
Comment thread tools/oracle-core/src/abi.rs
Comment thread tools/oracle-core/src/state.rs Outdated
Comment thread tools/oracle-core/src/derive.rs
Comment thread tools/oracle-core/src/derive.rs
Comment thread tools/oracle-task/src/mlow.rs Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 9da4e594e2

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread .github/workflows/mlow-derive.yml
Comment thread tools/oracle-core/src/wasi.rs Outdated
Comment thread tools/oracle-core/src/derive.rs
Comment thread tools/oracle-core/src/runtime.rs Outdated
Comment thread tools/oracle-core/src/emscripten.rs Outdated
Comment thread tools/oracle-core/src/media_probe.rs
Comment thread tools/oracle-core/src/call.rs Outdated
Comment thread tools/oracle-core/src/shared.rs Outdated
Comment thread tools/oracle-task/src/oracle_patch.rs Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 105d6c3d70

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread tools/oracle-core/src/wasi.rs Outdated
Comment thread tools/oracle-core/src/wasi.rs Outdated
Comment thread tools/oracle-core/src/wasi.rs
Comment thread tools/oracle-core/src/wasi.rs Outdated
Comment thread tools/oracle-core/src/wasi.rs Outdated
Comment thread tools/oracle-core/src/snapshot.rs

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a0093fbd2f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread tools/oracle-core/src/wasi.rs Outdated
Comment thread tools/oracle-core/src/wasi.rs
Comment thread tools/oracle-core/src/wasi.rs Outdated
Comment thread tools/oracle-core/src/wasi.rs Outdated
Comment thread tools/oracle-task/src/mlow.rs Outdated
Comment thread tools/oracle-core/src/threads.rs
Comment thread tools/oracle-core/src/emscripten.rs
Comment thread tools/oracle-core/src/data.rs Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d558404982

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread tools/oracle-core/src/threads.rs Outdated
Comment thread tools/oracle-core/src/wasi.rs Outdated
Comment thread tools/oracle-core/src/call.rs Outdated
Comment thread tools/oracle-core/src/emval.rs Outdated
Comment thread tools/oracle-core/src/embind.rs Outdated
Comment thread tools/oracle-core/src/runtime.rs Outdated
Comment thread tools/oracle-core/src/patch.rs Outdated
Comment thread tools/oracle-core/src/data.rs

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 0bf70ba861

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread tools/oracle-core/src/wasi.rs Outdated
Comment thread tools/oracle-core/src/wasi.rs
Comment thread tools/oracle-core/src/wasi.rs
Comment thread tools/oracle-core/src/runtime.rs Outdated
Comment thread tools/oracle-task/src/oracle_patch.rs Outdated
Comment thread tools/oracle-core/src/cxa.rs Outdated
Comment thread tools/oracle-core/src/abi.rs
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@jlucaso1
jlucaso1 merged commit 965f020 into main Sep 6, 2026
37 of 39 checks passed
@jlucaso1
jlucaso1 deleted the mlow-wasm-vectors branch September 6, 2026 18:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants