Add Eisel-Lemire decimal parsing fast path - #3957
Conversation
d55deac to
8dd3284
Compare
Coverage Report for CI Build 6249Coverage increased (+0.02%) to 90.764%Details
Uncovered ChangesNo uncovered changes found. Coverage RegressionsNo coverage regressions found. Coverage Stats
💛 - Coveralls |
8dd3284 to
87dcf24
Compare
|
Rebased onto
Validation: Codex CLI review (ultra reasoning effort) — sign-off, no findings
Codex also ran an independent differential fuzz of the ported algorithm against CPython's float parsing — 1,000,000 random (mantissa, exponent, sign) cases biased across bit lengths and the full exponent range, 834,771 accepted by the fast path, zero mismatches — and confirmed the parser state handed to the fast path is exact whenever 🤖 Generated with Claude Code |
Whenever try_eisel_lemire64 accepts an input it must return the correctly rounded double, so both new strconv properties compare accepted results against the exact Decimal conversion, reached through parse_double by zero-padding the mantissa spelling past the 19-digit fast-path limit. One draws mantissas across every magnitude; the other clusters them around powers of two where rounding carries and halfway ambiguity live. The json property pins the scanner wiring: plain, fraction-padded, and pointed spellings of the same value must parse to the same bits, with the padded spelling forcing the exact fallback that anchors the other routes. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
87dcf24 to
9b159ba
Compare
There was a problem hiding this comment.
Pull request overview
This pull request introduces a correctly-rounded Eisel–Lemire fast path for decimal-to-binary64 parsing in the internal/strconv standard-library package, and wires JSON number lexing to feed pre-scanned mantissa/exponent/sign data directly into that fast path to avoid expensive arbitrary-precision Decimal fallback for common long-mantissa inputs.
Changes:
- Add an Eisel–Lemire conversion implementation plus a generated cached-power table to
internal/strconv, keeping NaN as a private “reject” sentinel so the exact Decimal fallback remains authoritative for ambiguous/edge cases. - Extend
parse_doubleto try Eisel–Lemire when Clinger’s fast path doesn’t apply and the scanned digits were not truncated. - Update JSON number lexing to attempt Eisel–Lemire using lexer-scanned mantissa/exponent data; add targeted regression tests, property tests, and benchmarks for long-mantissa cases.
Reviewed changes
Copilot reviewed 11 out of 11 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| NOTICE | Records adaptation provenance for the new Eisel–Lemire implementation and table from Go 1.26.2 sources. |
| json/number_quickcheck_test.mbt | Adds a property test ensuring JSON number parsing is invariant to equivalent spellings (fast paths vs exact fallback). |
| json/number_bench_test.mbt | Adds a benchmark case focused on long-mantissa JSON number arrays (Canada-style coordinates). |
| json/lex_number.mbt | Routes JSON number lexing through Eisel–Lemire when Clinger doesn’t apply and digits aren’t truncated. |
| json/lex_number_test.mbt | Adds regression tests validating exact bit patterns for specific 17–19 digit mantissa cases. |
| internal/strconv/strconv_eisel_lemire.mbt | Implements try_eisel_lemire64 with explicit rejection of uncertifiable rounding cases to preserve correctness via fallback. |
| internal/strconv/strconv_eisel_lemire_wbtest.mbt | Adds whitebox tests comparing accepted Eisel–Lemire results against exact Decimal conversion and covering boundaries. |
| internal/strconv/strconv_eisel_lemire_table.mbt | Adds the generated cached-power table required by Eisel–Lemire for efficient scaling. |
| internal/strconv/strconv_double.mbt | Integrates the Eisel–Lemire attempt into parse_double after Clinger’s fast path and before Decimal fallback. |
| internal/strconv/parse_double_bench_test.mbt | Adds a long-mantissa parsing benchmark for parse_double. |
| internal/strconv/eisel_lemire_quickcheck_test.mbt | Adds property tests ensuring any Eisel–Lemire-accepted value matches exact Decimal rounding (bit-for-bit). |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Summary
internal/strconvNOTICEMotivation
Canada coordinate values such as
-65.613616999999977have 17-19 digit mantissas that exceed the exact Clinger limit. Nearly every coordinate therefore fell back to arbitrary-precision Decimal conversion, making decimal parsing the dominant JSON decode cost.The new path handles those mantissas with cached powers and explicit ambiguity checks. Inputs that cannot be certified by the fast path continue through the existing exact conversion.
Performance
Re-measured after the rebase onto
main, per backend (Apple Silicon macOS,moon bench, mean of 10×N runs):parse_double, long mantissas (4,096 parses):JSON long-mantissa array (10,000 numbers):
Existing integer and short-float paths remained within benchmark noise on all three backends (fast-digit, underscore, JSON int-array, and JSON float-array benches all within a few percent).
Validation
moon test internal/strconv json --target all: 228/228 passed on native, JS, Wasm, and Wasm-GCmoon check --target all --warn-list +73moon infoandmoon fmtgit diff --check.mbtichangesA full
moon test --target allrun reached an unrelated host-environment failure in the builtin Wasm tests becausewasi_snapshot_preview1.random_getwas not callable. The affectedinternal/strconvandjsonpackages pass independently on Wasm.