perf(string): accelerate ASCII character sets

[mizchi]

Summary

• Represent multi-character ASCII sets with four scalar UInt bitmaps.
• Route contains_any, trim_start, trim_end, and trim through the bitmap scan.
• Preserve the existing contains_char implementation for sets containing non-ASCII characters.
• Add regression coverage and benchmarks for long inputs.

Why

The previous multi-character paths scanned chars for every character in the input (O(n × m)). ASCII delimiter sets are common in trimming and token scanning; building the set once makes the scan O(n + m).

The bitmap is passed as scalar words, so the optimized path introduces no temporary heap allocation.

Benchmarks

Input size: 100,000 characters. Lower is better.

Benchmark	native before	native after	wasm-gc before	wasm-gc after
contains_any ASCII miss	180.87 µs	73.35 µs	301.89 µs	109.85 µs
contains_any match at end	194.44 µs	72.90 µs	301.19 µs	109.80 µs
trim_start	289.35 µs	198.22 µs	766.30 µs	544.25 µs
trim_end	260.92 µs	195.15 µs	858.34 µs	626.28 µs
trim	552.72 µs	375.94 µs	1.59 ms	1.14 ms

Validation

• moon test builtin/string_methods.mbt --target native --quiet
• moon test builtin/string_methods.mbt --target wasm-gc --quiet
• moon test builtin/string_methods.mbt --target js --quiet
• moon check builtin --target all
• moon info builtin
• Native and wasm-gc benchmarks in builtin/string_char_set_bench_test.mbt

No public API changes.

[Copilot]

Pull request overview

This PR optimizes common string character-set operations by introducing an ASCII fast path that uses a compact bitmap representation, improving performance for typical delimiter/whitespace trimming and scanning workloads in the MoonBit standard library.

Changes:

• Added a 128-bit ASCII character-set bitmap representation (four scalar UInts) and helper routines to test membership and scan/traverse strings efficiently.
• Routed StringView::contains_any, trim_start, trim_end, and trim through the ASCII bitmap path when the provided character set is ASCII-only, while preserving the existing non-ASCII fallback behavior.
• Added regression coverage in builtin/string_methods.mbt and introduced new benchmarks in builtin/string_char_set_bench_test.mbt.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

File	Description
builtin/string_methods.mbt	Implements ASCII bitmap fast paths for contains_any/trim* and adds regression tests to validate ASCII vs non-ASCII behavior.
builtin/string_char_set_bench_test.mbt	Adds benchmarks for contains_any and trimming on long inputs to quantify performance improvements.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[bobzhang]

Reviewed the bitmap approach and pushed two follow-up commits (9741cf28, 1e809040) that SIMD-accelerate the scans and close the gaps the 100k-char benchmarks didn't cover.

Review of the bitmap baseline

Correctness is solid: the ASCII bitmap is equivalent to contains_char for ASCII sets (a set char ≥ 0x80 falls back to the old path; an ASCII code unit is never half of a surrogate pair, so trim boundaries can't split a pair). Two gaps:

1. Short-input regression: default trim() on short strings was ~20% slower than main (the per-call set build isn't amortized; only 100k-char inputs were benchmarked).
2. The scans still decoded characters; for ASCII-only sets, scanning raw code units is provably equivalent and cheaper — and is the door to SIMD.

What the follow-up commits do

• SIMD scan on native/wasm: builtin can't import @v128 (cycle), but builtin/simd.mbt already hosts private #intrinsic shims with portable fallbacks for the existing string scanners — this reuses them (one new v128_or shim), scanning 8 code units per block and comparing against every set member broadcast to a vector (sets of ≤ 8 members; larger sets keep the scalar bitmap).
• Code-unit scalar scans everywhere else, except js contains_any, which keeps the character iterator (V8 compiles the method-shaped iterator loop measurably faster; verified by A/B).
• #inline on build_ascii_char_set removes the short-input overhead.
• Quickcheck properties (blackbox, all backends): contains_any/trims vs a character-by-character model on adversarial strings and offset views; the fused trim vs composed one-sided trims; an exhaustive sweep placing set members at every position across SIMD block boundaries for SIMD-sized, bitmap-only-sized, and default-whitespace sets; lone-surrogate cases; short-input benchmarks.

Benchmarks (Apple Silicon; before = PR head 4dc84002, same bench file)

Benchmark	target	before	after	Change
contains_any miss n=100000	native	76.40 µs	14.60 µs	5.2x
trim_start n=100000	native	307.02 µs	14.72 µs	20.9x
trim_end n=100000	native	244.57 µs	14.70 µs	16.6x
trim n=100000	native	565.59 µs	29.46 µs	19.2x
trim short inputs	native	149.92 ns	108.90 ns	1.4x
contains_any miss n=100000	wasm-gc	140.10 µs	99.43 µs	1.4x
trim_start n=100000	wasm-gc	591.40 µs	105.91 µs	5.6x
trim n=100000	wasm-gc	1.36 ms	216.61 µs	6.3x
trim short inputs	wasm-gc	209.73 ns	89.84 ns	2.3x
contains_any miss n=100000	js	211.32 µs	211.16 µs	parity
trim_start n=100000	js	522.51 µs	351.60 µs	1.5x
trim n=100000	js	977.83 µs	724.59 µs	1.3x
trim short inputs	js	198.08 ns	94.71 ns	2.1x

Short default trim() is now also faster than pre-PR main (108.9 ns vs ~114 ns), so the short-input regression is gone. The only measured cost is js short contains_any at 83.7 ns vs 78.6 ns (~1.2 ns/call).

Validation

• moon fmt, moon info (no public API changes), moon check --target all
• moon test -p moonbitlang/core/builtin --target all: 2957 (wasm) / 2957 (wasm-gc) / 2934 (js) / 2916 (native), all passing

[coveralls]

Coverage Report for CI Build 6250

Warning

Build has drifted: This PR's base is out of sync with its target branch, so coverage data may include unrelated changes.
Quick fix: rebase this PR. Learn more →

Coverage increased (+0.02%) to 90.763%

Details

• Coverage increased (+0.02%) from the base build.
• Patch coverage: 9 uncovered changes across 2 files (135 of 144 lines covered, 93.75%).
• No coverage regressions found.

Uncovered Changes

File	Changed	Covered	%
builtin/string_methods.mbt	143	135	94.41%
builtin/simd.mbt	1	0	0.0%

Coverage Regressions

No coverage regressions found.

---

Coverage Stats

Relevant Lines:	18123
Covered Lines:	16449
Line Coverage:	90.76%
Coverage Strength:	329566.38 hits per line

---

💛 - Coveralls

[bobzhang]

Follow-up on the review loop for the SIMD commits.

Round 1 (Codex CLI, codex exec review --base origin/main, model_reasoning_effort="ultra") found a real P1: an empty chars set still entered the SIMD trim path, and the splat helper read one code unit past the set view — an empty view carved from a longer string ("b"[0:0]) broadcast the backing b as a set member, so ("b".repeat(8) + "x").trim_start(chars=empty) returned "x" on native/wasm. Fixed in aeac20f5: the SIMD scans now require a non-empty set (no measurable cost), with a direct regression test for empty sets/views, and the quickcheck properties now pass every generated set through an offset view with sentinel neighbors so any future out-of-view read shows up as a model disagreement.

Round 2 re-review came back clean. Its signoff:

The ASCII bitmap and SIMD paths preserve StringView bounds and correctly handle empty, non-ASCII, and surrogate-containing inputs. Targeted tests passed across native, wasm, wasm-gc, and JavaScript backends.

It also benchmarked the wasm (linear) backend independently, where the SIMD path is active as well: contains_any 13.2 µs, trim_start/trim_end 14.8 µs, trim 29.5 µs on the 100k-char inputs — same class as native.

Final state: moon check --target all clean, builtin suite 2958 (wasm) / 2958 (wasm-gc) / 2935 (js) / 2917 (native) all passing, moon fmt / moon info no drift, no public API changes.