perf(wasm): scope ensureWasmTrees re-parse to files that need it

[carlos-alm]

Summary

Fixes #1036 — WASM full build regressed from 7.6s (3.9.5) to 14.0s (3.9.6) on the 744-file dogfooding corpus.

Root cause: PR #1016 expanded AST_TYPE_MAPS from 3 to 23 languages, growing WALK_EXTENSIONS to cover .rs/.go/.py/etc. Files like crates/codegraph-core/build.rs (5 lines, no strings/awaits/throws) produce zero ast_nodes, so the worker returned astNodes: undefined. On the main thread, fileNeedsWasmTree saw !Array.isArray(symbols.astNodes) && WALK_EXTENSIONS.has('.rs') and flagged the file as needing re-parse — at which point ensureWasmTrees ignored the per-file decision and re-parsed every WASM-parseable file in the build.

Fix (3 surgical changes):

1. wasm-worker-entry.ts — always serialize astNodes as an array (even empty) when ast-store ran for the file. Empty ≠ undefined: empty means "we walked it and found nothing", which is what fileNeedsWasmTree needs to see.
2. parser.ts::ensureWasmTrees — accept an optional needsFn filter so the caller can scope re-parse to files that genuinely lack data, instead of pulling in every WASM-parseable file in the map.
3. ast-analysis/engine.ts — pass fileNeedsWasmTree as that filter.

Bonus: rolled in two small ast-store-visitor optimizations found while profiling — hoist the newTypes Set into a per-astTypeMap WeakMap cache (was rebuilt per file), and skip the findParentDef linear scan when nodeIdMap is empty (worker context — main thread re-resolves anyway). Codepoint check uses an s.length-based fast path that only spreads when length 2 or 3 needs surrogate-pair disambiguation.

Bench (744 files, dogfooding)

Metric	Before (3.9.6)	After	Target
WASM full build	14014 ms	7847 ms	<9000 ms ✓
Native full build	1693 ms	1704 ms	≤6000 ms ✓
WASM incremental	51 ms	51 ms	unchanged ✓
AST nodes stored (WASM)	39702	39702	parity with native ✓

Test plan

• WASM full build < 9s (7.8s, restores 3.9.5 baseline)
• Native full build ≤ 6s (1.7s, no regression)
• Incremental rebuild correctness preserved (oneFileRebuildMs unchanged)
• ast_nodes table populated correctly on WASM build (39702 rows: string/new/await/regex/throw)
• npm run build (TypeScript) succeeds

[claude]

Claude finished @carlos-alm's task —— View job

---

PR Review: Performance Optimization for WASM AST Store

Review Checklist:

• Analyze root cause and approach validation
• Check for configuration change red flags
• Evaluate test integrity
• Review code quality and implementation
• Assess backlog compliance criteria
• Provide structured final assessment

Root Cause Analysis

Problem Being Solved: Performance regression in WASM builds (7.6s → 14.0s) introduced in v3.9.6 when PR #1016 expanded AST_TYPE_MAPS from 3 to 23 languages. Files with zero AST nodes (like build.rs) returned astNodes: undefined from worker, causing fileNeedsWasmTree to trigger full re-parse of ALL WASM-parseable files instead of just the specific file.

Why This Approach: Three surgical fixes target the exact bug - worker always returns astNodes: [] (not undefined), ensureWasmTrees accepts filtering function, and caller passes precise filter. No architectural changes, maintains exact same API contracts.

Risk Assessment: Very low - changes are scoped, preserve existing behavior, and the performance regression is well-characterized with clear benchmarks.

Backlog Compliance

• Zero-dep: ✓ — Pure performance optimization using existing tree-sitter/SQLite infrastructure
• Foundation-aligned: ✓ — Directly supports P1 (always-current graph) by making WASM rebuilds fast enough for commit hooks
• Problem-fit: 5/5 — Eliminates 6.2s of wasted rebuild time, directly enabling tighter agent loops and commit hooks
• Breaking: No — Pure internal optimization, no API or behavioral changes

Critical Concerns

None. The fixes are technically sound and target the exact root cause:

1. Line 711-722 in wasm-worker-entry.ts: Critical semantic fix - always serialize astNodes as [] instead of undefined
2. Line 325 in parser.ts: Add optional needsFn parameter for targeted re-parsing
3. Lines 424-426 in engine.ts: Pass fileNeedsWasmTree as filter function

The bonus optimizations in ast-store-visitor.ts are well-engineered:

• newTypesCache WeakMap (lines 171-181): Eliminates redundant Set rebuilding per file
• skipParentLookup check (lines 195-198): Skips linear scan when nodeIdMap is empty
• codePointCountAtLeast2 (lines 141-146): Fast-path Unicode counting optimization

All changes maintain behavioral parity while eliminating redundant work.

Final Recommendation

• Rating: ⭐⭐⭐⭐⭐ (5/5)
• Action: APPROVE
• Reasoning: Excellent root cause analysis, surgical implementation targeting exact bug, significant performance improvement (14.0s → 7.8s), maintains API compatibility, includes thoughtful micro-optimizations, and directly supports core principle P1 (always-current graph).

[greptile-apps]

Greptile Summary

This PR fixes a WASM full-build regression (#1036) where ensureWasmTrees re-parsed every WASM-parseable file whenever any single file lacked AST data. The three-part fix — always serializing astNodes as an array in the worker, adding a needsFn filter to ensureWasmTrees, and passing fileNeedsWasmTree as that filter — is logically correct and well-targeted. The bonus ast-store-visitor optimizations (WeakMap-cached newTypes, skipped findParentDef when nodeIdMap is empty, codePointCountAtLeast2 fast path) are all correct.

Confidence Score: 5/5

This PR is safe to merge — it fixes a well-diagnosed performance regression with surgical, well-tested changes and no correctness regressions.

All changes are logically correct: the undefined→empty-array fix is consistent with mergeAnalysisData and fileNeedsWasmTree, the needsFn filter is backward-compatible (optional parameter, identical fallback behavior), and the ast-store-visitor micro-optimizations are mathematically sound. Benchmark data and regression-guard entries substantiate the fix.

No files require special attention.

Important Files Changed

Filename	Overview
src/domain/wasm-worker-entry.ts	Always sets astNodes to an array (even empty) when astVisitor ran, fixing the root-cause undefined-vs-empty confusion that triggered blanket re-parses.
src/domain/parser.ts	Adds optional needsFn to ensureWasmTrees so callers can scope re-parse to files that genuinely need it; fallback (no filter) preserves backward compatibility.
src/ast-analysis/engine.ts	Threads fileNeedsWasmTree as the needsFn filter into ensureWasmTrees; the two-pass scan (any-needs-wasm + per-file filter) is consistent and correct.
src/ast-analysis/visitors/ast-store-visitor.ts	Three performance micro-optimizations: WeakMap cache for newTypes, skipParentLookup guard, and codePointCountAtLeast2 fast path — all logically correct.
tests/benchmarks/regression-guard.test.ts	Adds 5 well-documented KNOWN_REGRESSIONS entries for 3.9.6 data captured before the fix, with clear explanations for each skip.

Sequence Diagram

sequenceDiagram
    participant E as engine.ts
    participant P as parser.ts ensureWasmTrees
    participant W as wasm-worker-entry.ts

    E->>E: scan fileSymbols, fileNeedsWasmTree() any true?
    E->>P: ensureWasmTrees(fileSymbols, rootDir, needsFn)
    loop for each relPath in fileSymbols
        P->>P: skip if _tree exists
        P->>P: skip if ext not in _extToLang
        P->>E: needsFn(relPath, symbols)?
        E-->>P: fileNeedsWasmTree(relPath, symbols, flags)
        alt needsFn returns true
            P->>W: pool.parse(absPath, code, FULL_ANALYSIS)
            W->>W: walkWithVisitors()
            W->>W: astRows = results[ast-store]
            Note over W: Always serialize as array (even empty) when astVisitor ran
            W-->>P: astNodes: [] or [...rows]
            P->>P: mergeAnalysisData(symbols, output)
        else needsFn returns false
            P->>P: skip file (already has data)
        end
    end

Loading

_{Reviews (2): Last reviewed commit: "test(bench): mark known 3.9.6 regression..." | Re-trigger Greptile}

[greptile-apps]

len === 3 spread is always redundant

A UTF-16 string of length 3 must contain at least 2 code points (worst case: one surrogate pair + one regular char = 2 code points; all other combinations give ≥ 3). The [...s] spread for len === 3 always evaluates to true, so you can short-circuit it with an early return, keeping the spread only for the ambiguous len === 2 case.

Suggested change

	if (len >= 4) return true;
	return [...s].length >= 2;
	if (len >= 3) return true;
	return [...s].length >= 2;

[carlos-alm]

Fixed in c0a089f — folded the redundant len===3 case into the fast path so the spread only runs for the genuinely ambiguous len===2 case. Updated the comment to spell out the worst-case reasoning (1 surrogate pair + 1 BMP char = 2 code points).

[github-actions]

Codegraph Impact Analysis

8 functions changed → 16 callers affected across 5 files

• ensureWasmTreesIfNeeded in src/ast-analysis/engine.ts:399 (3 transitive callers)
• codePointCountAtLeast2 in src/ast-analysis/visitors/ast-store-visitor.ts:142 (5 transitive callers)
• extractStringContent in src/ast-analysis/visitors/ast-store-visitor.ts:154 (7 transitive callers)
• newTypesFor in src/ast-analysis/visitors/ast-store-visitor.ts:173 (7 transitive callers)
• createAstStoreVisitor in src/ast-analysis/visitors/ast-store-visitor.ts:184 (9 transitive callers)
• resolveParentNodeId in src/ast-analysis/visitors/ast-store-visitor.ts:213 (2 transitive callers)
• ensureWasmTrees in src/domain/parser.ts:325 (3 transitive callers)
• handleParse in src/domain/wasm-worker-entry.ts:653 (1 transitive callers)