perf(hex): vectorize encode/decode on the linear-memory backends

[bobzhang]

Follow-up to #3625, which landed encoding/hex with a scalar-only codec. This adds v128 fast paths for encode and decode on the linear-memory backends, structured exactly like the sibling encoding/base64: the scalar codec stays on every backend as the reference implementation, and the public entry points dispatch with #cfg.

How it works

Encode — 16 source bytes per iteration. Split each byte into its nibbles (i8x16_shr_u and a mask), interleave them into output order with two i8x16_shuffles, and convert nibbles to ASCII with one register-resident i8x16_swizzle table. A MoonBit String is UTF-16, so the 32 characters are widened with i16x8_extend_low/high_i8x16_u before being stored.

Decode — 32 characters per iteration. Four v128_load_i16x8 are narrowed pairwise with i8x16_narrow_i16x8_u; that instruction reads its lanes as signed and saturates, so a code unit in 0x0100..=0x7FFF becomes 255 and one in 0x8000..=0xFFFF (surrogate halves included) becomes 0. Neither is a hex digit, so no non-ASCII code unit can masquerade as one by way of its low byte. The three hex ranges are then validated vectorially and folded to nibbles with wrapping adds. Anything the fast path will not take — short, odd-length, or invalid input — falls back to decode_scalar, which remains the single authority on the Malformed rules, so decode raises on exactly the inputs it did before.

Performance

moon bench encoding/hex, 4 KiB payload, scalar path vs. the public entry point on the same run:

target	encode scalar	encode	speedup	decode scalar	decode	speedup
native	16.33 µs	634 ns	25.8x	7.56 µs	815 ns	9.3x
wasm	35.64 µs	947 ns	37.6x	21.17 µs	1.33 µs	15.9x
js	38.60 µs	35.93 µs	1.07x	43.38 µs	44.01 µs	0.99x

js keeps the scalar path by #cfg, so both columns measure the same code and confirm no regression there. Other sizes, public entry point: 64 B encode is 34.5 ns native / 94.5 ns wasm / 608 ns js; 64 KiB is 16.0 µs / 15.7 µs / 687 µs encode and 12.7 µs / 20.5 µs / 746 µs decode.

Correctness

Differential white-box tests pin both fast paths against the scalar ones: every length from 0 to 79 (so every block count and tail residue), views that start and end inside a larger buffer whose padding is itself valid hex, an invalid character planted at every lane position, and raw UTF-16 code units — lone surrogates included — planted at every position of a full block. A #cfg-gated test calls encode_v128 and decode_v128 directly, so the suite fails if the fast path is ever silently disabled rather than exercised.

Four quickcheck properties cover decode∘encode against an independent reference encoder, case-mixture round-trips, corruption at any position raising Malformed, and odd-length input raising Malformed.

Each of these mutations makes the suite fail, which is how I know the tests discriminate: an interleave shuffle index, the -7 nibble adjustment constant, either start_offset() replaced by 0, the saturating narrow replaced by a truncating shuffle, and decode_v128 returning None unconditionally.

Both block loops bound themselves by subtraction, and the encoder declines an input whose UTF-16 byte buffer would not fit in an Int, so no vector store can be reached through a wrapped index.

moon test passes 7512/7512; encoding/hex is 20/20 on native and wasm, 19/19 on wasm-gc and js. pkg.generated.mbti is unchanged — this is an implementation change only.

Review

Reviewed by Codex CLI at ultra reasoning effort over two rounds. Round 1 withheld approval over two P1 memory-safety findings — a wrapping length * 4 output size and a index + 16 block bound that could wrap for a view near Int::MAX — plus P2s on test discrimination and a P3 documentation error about the narrow's signed-lane behaviour. All are fixed; round 2 verdict and sign-off are posted in a comment below.

[bobzhang]

Codex CLI review (ultra reasoning effort)

Two rounds. Round 1 withheld approval; round 2 approved after the fixes.

Round 1 — not approved

I would not approve this change yet. Two P1 memory-safety issues need fixing.

• P1 — output-size overflow causes unchecked SIMD stores. In encode_v128.mbt, length * 4 uses wrapping 32-bit arithmetic. At length == 1 << 30, it becomes zero, but the loop subsequently writes four 16-byte vectors into that zero-length output. Generated native code confirms these are raw, bounds-unchecked SIMD stores. Even when the requested result is too large to represent, this must fail safely rather than corrupt memory. Use checked sizing and fall back or abort before allocation.

• P1 — the absolute view index can overflow the block guard. In encode_v128.mbt, index + 16 can wrap negative for a short suffix view near Int::MAX. An eight-byte view beginning at Int::MAX - 15, for example, enters the SIMD loop, reads beyond the view/backing buffer, and writes 64 bytes into a 32-byte output. Drive the loop using relative consumed length or a subtraction-based bound such as end - index >= 16. The base64 precedent shares this defect; that does not make it safe.

• P2 — latent equivalent overflow in decode. decode_v128.mbt has the same index + 32 pattern. Near Int::MAX, it can wrap and process beyond the view. Current safe UTF-16 constructors cannot practically create such a large string, so this is hardening rather than an immediately reachable P1.

• P2 — tests do not prove either fast path remains enabled. v128_wbtest.mbt only exercises public entry points. Making decode_v128 always return None, or dispatching encode directly to encode_scalar, leaves every correctness test passing.

• P2 — malformed non-ASCII coverage misses aliasing cases. The pool in quickcheck_test.mbt omits NUL, lone surrogates, and code units whose low byte is a valid hex character. Replacing saturating narrowing with low-byte truncation would survive the current corpus but incorrectly accept values such as U+0130.

• P3 — narrowing documentation is inaccurate. decode_v128.mbt says every code unit above 255 becomes 255. This instruction interprets source i16 lanes as signed: 0x0100..0x7fff becomes 0xff, while 0x8000..0xffff, including surrogates, becomes zero. Both remain correctly rejected, so this is documentation-only.

Aside from the overflow paths, the SIMD reasoning checks out: both shuffle directions, swizzle table, wrapping constants, lowercase mask, recombination, UTF-16 widening, ordinary view bounds, and casts are correct. Modeling all 65,536 UTF-16 code units found no acceptance or decoded-value mismatch with the scalar decoder.

What changed in response

• Encode refuses an input whose UTF-16 byte buffer would not fit in an Int (length <= 0x1FFF_FFFF) and hands it to the scalar encoder, which builds the string without ever materializing that buffer. StringBuilder clamps a non-positive size_hint to capacity 1 and bounds-checks growth, so the fallback fails safely rather than relocating the problem.
• Both block loops now bound by subtraction — end - index >= 16 and length - index >= 32. Neither subtraction can wrap: end = base + length <= src.length() <= Int::MAX, and the index stays within the view.
• Added a #cfg-gated test calling encode_v128 and decode_v128 directly, so a silently disabled fast path fails the suite.
• Added a test planting raw UTF-16 code units — 0x0130, 0x0161, 0xD830, 0xDC30, 0xFF41, 0x1030, plus 0x0000 and 0x00FF — at every position of a full block. All but the last two alias a hex digit in their low byte; replacing the saturating narrow with a truncating shuffle now fails.
• Corrected the narrowing documentation to state the signed-lane behaviour.

Round 2 — approved

No P0/P1 findings. I approve.

• The length * 4 guard prevents allocation and store-offset overflow. The scalar fallback is safe: wrapped hints are only capacity hints, writes are checked, and growth detects overflow.
• No previously successful input regresses; the unchanged scalar implementation handles sizes rejected by SIMD.
• Both subtraction-based loops process exactly the same full blocks and tails while preventing index wrap.
• The signed-lane narrowing documentation is correct, including surrogate handling.
• Direct fast-path and malformed-code-unit tests adequately cover the fixes.

Two non-blocking comment nits:

• encode_v128.mbt: inputs through Int::MAX / 2 can have a representable String; only the SIMD byte buffer exceeds Int::MAX.
• v128_wbtest.mbt: 0x0000 is neither non-ASCII nor a low-byte hex alias. The other six aliasing values still make the test discriminating.

Current artifacts passed 20/20 on native and wasm and 19/19 on wasm-gc and JS; formatting and generated interface artifacts match.

Signed-off-by: Codex CLI codex@openai.com

Both nits were comment-only and are fixed in the pushed branch.

One finding reaches beyond this PR: encoding/base64 shares the wrapping index + 16 block bound that round 1 flagged as P1 here. It is equally unreachable in practice and equally free to fix; I will send it as a separate patch rather than widen this one.

[Copilot]

Pull request overview

This PR adds SIMD (v128) fast paths for encoding/hex on the linear-memory backends (native + wasm), while preserving the scalar implementation as the reference and fallback. It mirrors the structure used by encoding/base64, using #cfg to dispatch at the public entry points.

Changes:

• Add encode_v128 / decode_v128 implementations and dispatch encode/decode to them on #cfg(any(target="native", target="wasm")).
• Add differential white-box tests plus QuickCheck properties to pin vector behavior against scalar behavior and error rules.
• Add white-box benchmarks to measure scalar vs public entry points side-by-side.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
encoding/hex/encode.mbt	Adds #cfg dispatch so encode uses scalar on non-linear-memory targets and v128 on native/wasm.
encoding/hex/encode_v128.mbt	New v128 encoder implementation for 16-byte blocks with UTF-16 widening stores.
encoding/hex/decode.mbt	Adds #cfg dispatch so decode uses scalar on non-linear-memory targets and v128-with-fallback on native/wasm.
encoding/hex/decode_v128.mbt	New v128 decoder implementation for 32-code-unit blocks with vector validation + nibble folding.
encoding/hex/v128_wbtest.mbt	New differential white-box tests validating v128 paths vs scalar paths and view offset handling.
encoding/hex/quickcheck_test.mbt	New randomized property tests covering round-trips, case mixtures, corruption, and odd-length failure.
encoding/hex/v128_bench_wbtest.mbt	New white-box benchmarks comparing scalar vs public entry points at multiple sizes.
encoding/hex/moon.pkg	Adds v128 dependency plus test/wbtest imports consistent with SIMD-enabled encoding packages.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[coveralls]

Coverage Report for CI Build 6261

Coverage increased (+0.02%) to 90.795%

Details

• Coverage increased (+0.02%) from the base build.
• Patch coverage: 3 uncovered changes across 3 files (70 of 73 lines covered, 95.89%).
• No coverage regressions found.

Uncovered Changes

File	Changed	Covered	%
encoding/hex/decode.mbt	4	3	75.0%
encoding/hex/encode.mbt	2	1	50.0%
encoding/hex/encode_v128.mbt	23	22	95.65%
Total (4 files)	73	70	95.89%

Coverage Regressions

No coverage regressions found.

---

Coverage Stats

Relevant Lines:	18273
Covered Lines:	16591
Line Coverage:	90.8%
Coverage Strength:	306320.11 hits per line

---

💛 - Coveralls

[bobzhang]

Addressed the Copilot review in b77d659.

All three are correct. size_hint is measured in bytes and a UTF-16 code unit occupies two of them, so a hint derived from a code-unit count needs a factor of two — the factor of four is right only in reference_encode, where the count is source bytes and each becomes two code units. Copilot flagged exactly the four code-unit sites and left that one alone.

• v128_wbtest.mbt:133, :156 and quickcheck_test.mbt:50, :82: 4 * -> 2 *.
• v128_wbtest.mbt:48: grammar.

Tests still pass 20/20 on native and wasm, 19/19 on wasm-gc and js.