Deferred from PR #1107.
Context: Adding native Verilog support pulled the 4 .v resolution-benchmark
fixtures (tests/benchmarks/resolution/fixtures/verilog/*.v) into the corpus
that scripts/incremental-benchmark.ts builds (it walks the entire repo root).
tree-sitter-verilog is a large grammar, so parsing those fixtures adds ~850ms
to the native fullBuildMs measurement (1959 → 2809ms, +43% vs the 3.10.0
baseline). The PR landed with 3.10.0:Full build added to KNOWN_REGRESSIONS
so the per-PR benchmark gate is unblocked.
Fix options:
- Add an
exclude glob (e.g. tests/benchmarks/resolution/fixtures/**) to
the buildGraph invocation in scripts/incremental-benchmark.ts so fixture
files don't inflate the timing for any new-language PR going forward. This
is the principled fix — resolution-benchmark fixtures are not representative
of real codebases the benchmark is supposed to measure.
- Alternatively, accept the structural cost as the price of supporting Verilog
and remove the 3.10.0:Full build exemption once 3.11.0+ data establishes
the new baseline.
Once resolved, the 3.10.0:Full build entry in tests/benchmarks/regression-guard.test.ts
should be removed.
Originally raised by: PR #1107 CI failure (run 25716010487).
Deferred from PR #1107.
Context: Adding native Verilog support pulled the 4
.vresolution-benchmarkfixtures (
tests/benchmarks/resolution/fixtures/verilog/*.v) into the corpusthat
scripts/incremental-benchmark.tsbuilds (it walks the entire repo root).tree-sitter-verilog is a large grammar, so parsing those fixtures adds ~850ms
to the native
fullBuildMsmeasurement (1959 → 2809ms, +43% vs the 3.10.0baseline). The PR landed with
3.10.0:Full buildadded toKNOWN_REGRESSIONSso the per-PR benchmark gate is unblocked.
Fix options:
excludeglob (e.g.tests/benchmarks/resolution/fixtures/**) tothe buildGraph invocation in
scripts/incremental-benchmark.tsso fixturefiles don't inflate the timing for any new-language PR going forward. This
is the principled fix — resolution-benchmark fixtures are not representative
of real codebases the benchmark is supposed to measure.
and remove the
3.10.0:Full buildexemption once 3.11.0+ data establishesthe new baseline.
Once resolved, the
3.10.0:Full buildentry intests/benchmarks/regression-guard.test.tsshould be removed.
Originally raised by: PR #1107 CI failure (run 25716010487).