Skip to content

perf(ascii): vectorize decode on non-JS targets - #3838

Merged
bobzhang merged 5 commits into
moonbitlang:mainfrom
mizchi:perf/ascii-decode-v128
Aug 21, 2026
Merged

perf(ascii): vectorize decode on non-JS targets#3838
bobzhang merged 5 commits into
moonbitlang:mainfrom
mizchi:perf/ascii-decode-v128

Conversation

@mizchi

@mizchi mizchi commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Add a non-JS SIMD path for @ascii.decode.
  • Process sixteen input bytes at once: reject blocks with any high bit set, then zero-extend ASCII bytes into UTF-16 output lanes.
  • Keep the scalar path below the measured 64-byte crossover point and retain the existing JavaScript implementation.
  • Add whitebox coverage for non-zero views and an invalid byte inside a SIMD block.
  • Add decode benchmarks from 15 bytes to 1,000,000 bytes.

Correctness

The SIMD path uses i8x16_bitmask to locate the first byte with its high bit set. On failure it raises Malformed with the same remaining BytesView as the scalar decoder.

Benchmarks

moon bench -p encoding/ascii -f ascii_bench_test.mbt --target <target> --release --no-parallelize

Target n=100,000 before after n=1,000,000 before after
native 26.19 us ~10 us 255.93 us 77.65 us
wasm 50.70 us 19.65–21.41 us 532.81 us 144.01 us
wasm-gc 212.94 us 150.34 us 2.15 ms 1.40 ms

n=15, n=16, and n=32 remain on the scalar path and stayed within measurement variance of the baseline.

Validation

  • moon test -p encoding/ascii --target all --no-render
    • wasm: 10/10; wasm-gc: 10/10; js: 8/8; native: 10/10
  • moon check encoding/ascii --target all --no-render
  • moon info -p encoding/ascii
  • moon fmt --check for all changed files
  • git diff --check

The installed compiler rejects an unrelated pattern in builtin/iterator.mbt:789 on an unmodified upstream checkout. The test and check commands above used a temporary equivalent rewrite of that pattern; it is not included in this PR.

moon bench --profile is not available in the installed Moon CLI, so the validation uses release benchmarks and generated-C inspection instead.

@mizchi
mizchi marked this pull request as ready for review July 20, 2026 11:39
Copilot AI review requested due to automatic review settings July 20, 2026 11:39

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR optimizes encoding/ascii decoding on non-JS targets by adding a SIMD (v128) fast path while preserving the existing JavaScript implementation and scalar performance for small inputs.

Changes:

  • Add a non-JS SIMD implementation for @ascii.decode, with a 64-byte crossover to keep short inputs on the scalar path.
  • Add whitebox tests covering non-zero BytesView offsets and correct error slicing when an invalid byte occurs inside a SIMD block.
  • Add decode micro-benchmarks across a range of input sizes (15 bytes to 1,000,000 bytes).

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated no comments.

File Description
encoding/ascii/moon.pkg Adds core/v128 dependency for SIMD decode and core/bench for benchmark tests; gates a wbtest file off JS.
encoding/ascii/decode.mbt Introduces non-JS decode_scalar + decode_v128 and selects between them; keeps JS decode intact via #cfg.
encoding/ascii/decode_v128_wbtest.mbt Adds whitebox tests for view offsets + first-invalid-byte reporting within a SIMD block.
encoding/ascii/ascii_bench_test.mbt Adds benchmark tests for multiple sizes to validate the intended crossover behavior and speedups.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@mizchi
mizchi force-pushed the perf/ascii-decode-v128 branch from 9d12cb3 to 054fe0e Compare July 27, 2026 09:36
@bobzhang
bobzhang force-pushed the perf/ascii-decode-v128 branch from 054fe0e to 7793b06 Compare August 21, 2026 00:28
@bobzhang

Copy link
Copy Markdown
Contributor

Rebased onto latest main (clean, no conflicts) and pushed a follow-up commit (7793b06e) with property-based tests. Review notes and independently measured benchmarks below.

Review

The SIMD design is correct and matches the scalar decoder exactly:

  • i8x16_bitmask + ctz locates the first high-bit byte, and the raised Malformed view is byte-identical to the scalar decoder's (verified by property test, below).
  • The little-endian v128_store of i16x8_extend_low/high_i8x16_u lanes produces exactly the UTF-16 layout the scalar tail writes.
  • Views are handled via absolute offsets into the backing array; unaligned view starts work (v128_load has no alignment requirement).
  • The 64-byte crossover is justified by measurement: n=64 already wins on every SIMD backend, and n=15/16/32 stay at parity on the scalar path.
  • Keeping emulated SIMD on wasm-gc is empirically right (1.4–1.65x, measured below) — the block structure beats the per-byte pattern-match loop even without hardware SIMD.

Possible follow-up (not blocking): decode_lossy still uses the scalar loop and could reuse the same block scan.

Added tests (7793b06e)

  • quickcheck: decode/decode_lossy vs a byte-by-byte model on poisoned views — payloads embedded between 0xFF bytes just outside the view bounds, at all 17 view offsets, so any out-of-view read shows up as a wrong result.
  • quickcheck: the Malformed view equals the suffix starting at the first invalid byte.
  • quickcheck (whitebox, non-js): decode_v128 agrees with decode_scalar — same string on success, same remaining view on failure — on mixed valid/invalid payloads through unaligned views.
  • quickcheck (whitebox, all targets incl. js): finish_string truncation.
  • Exhaustive sweep: an invalid byte planted at every position for lengths 1–96 around the 16-byte block and 64-byte crossover boundaries.

All targets pass: 15 (wasm) / 15 (wasm-gc) / 12 (js) / 15 (native), moon check --target all with zero warnings, moon fmt / moon info clean.

Benchmarks (Apple Silicon, --release; before = main e50d33d2 with the same bench file)

Target n=100,000 before after n=1,000,000 before after Change
native 27.57 µs 11.53 µs 274.82 µs 80.03 µs 2.4–3.4x
wasm 75.47 µs 15.16 µs 759.01 µs 149.31 µs 5.0–5.1x
wasm-gc 236.47 µs 166.59 µs 2.35 ms 1.55 ms 1.4–1.5x
js 478.89 µs 473.83 µs 5.48 ms 5.46 ms parity (path unchanged)

At the crossover, n=64: native 31.1 → 26.6 ns, wasm 86.7 → 64.4 ns, wasm-gc 167.1 → 101.2 ns; n=15/16/32 are at parity on every backend (scalar path retained).

@bobzhang

Copy link
Copy Markdown
Contributor

Ran the full branch through Codex CLI review (codex exec review --base origin/main, model_reasoning_effort="ultra"). It came back clean on the first round — no findings. Its signoff:

The SIMD decoder preserves scalar behavior for valid and malformed inputs, including sliced and unaligned views. Targeted checks and tests passed on wasm-gc, wasm, native, and JavaScript backends.

Its probing included the Malformed view semantics on both decode paths, the length * 2 buffer-sizing pattern against the analogous sites elsewhere in the tree (encoding/utf8, encoding/utf16, builtin/string.mbt), and independent multi-backend test runs.

With the earlier comment's rebase, property tests, and 4-backend benchmark verification, this looks ready from my side.

mizchi and others added 5 commits August 21, 2026 08:41
Add quickcheck properties pinning decode/decode_lossy against a
byte-by-byte model on poisoned views (invalid bytes just outside the
view bounds), the Malformed view against the first invalid byte, the
SIMD decoder against the scalar decoder on arbitrary payloads and
unaligned view offsets, and finish_string truncation; plus an
exhaustive invalid-byte position sweep around SIMD block and crossover
boundaries.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@bobzhang
bobzhang force-pushed the perf/ascii-decode-v128 branch from 7793b06 to 7e84e0e Compare August 21, 2026 00:41
@bobzhang
bobzhang enabled auto-merge (rebase) August 21, 2026 00:41
@bobzhang
bobzhang merged commit 4e89315 into moonbitlang:main Aug 21, 2026
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants