perf: avoid parsing concatenated module codegen#14852
Draft
LingyuCoder wants to merge 9 commits into
Draft
Conversation
Contributor
📦 Binary Size-limit
❌ Size increased by 152.00KB from 66.64MB to 66.79MB (⬆️0.22%) |
Contributor
Rsdoctor Bundle Diff AnalysisFound 5 projects in monorepo, 0 projects with changes. 📊 Quick Summary
Generated by Rsdoctor GitHub Action |
Contributor
Merging this PR will improve performance by 14.17%
|
| Mode | Benchmark | BASE |
HEAD |
Efficiency | |
|---|---|---|---|---|---|
| ⚡ | Simulation | bundle@threejs-production-sourcemap |
192.9 ms | 155 ms | +24.51% |
| ⚡ | Simulation | rust@concatenate_module_code_generation |
45.1 ms | 39.1 ms | +15.33% |
| ⚡ | Simulation | bundle@misc-production-sourcemap |
3.7 s | 3.5 s | +3.65% |
Tip
Curious why this is faster? Comment @codspeedbot explain why this is faster on this PR, or directly use the CodSpeed MCP with your agent.
Comparing seal/concat-local-ident (c1aa22c) with main (a5dfc7e)
Footnotes
-
47 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩
Contributor
Contributor
|
📝 Rspack Ecosystem CI: Open
|
Contributor
Deploying rspack with
|
| Latest commit: |
5432a85
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://98671a9e.rspack-v2.pages.dev |
| Branch Preview URL: | https://seal-concat-local-ident.rspack-v2.pages.dev |
Contributor
Author
CodSpeed Optimization Log
Final gates for exact head
|
Contributor
Contributor
There was a problem hiding this comment.
Pull request overview
This PR introduces an opt-in experiment (experiments.fasterModuleConcatenation, default false) to speed up module concatenation by reusing identifier/scope information captured during parsing and avoiding a second parse during concatenated-module code generation. It propagates the experiment flag through JS/Rust config surfaces, updates concatenation/name-deconflict mechanics across multiple generators/templates, and adds targeted test/benchmark coverage for the new path.
Changes:
- Add the
experiments.fasterModuleConcatenationoption end-to-end (JS config types/defaults → binding API → Rust core options/builders) and enable it in relevant benchmarks. - Capture and persist a compact concatenation-scope snapshot during the existing parser traversal, then use it during concatenated-module and ESM-library codegen to deconflict identifiers without reparsing.
- Update multiple dependency templates/generators (JS/JSON/CSS/assets/externals, etc.) to cooperate with the new faster concatenation mechanism, plus new/updated test cases and snapshots.
Reviewed changes
Copilot reviewed 73 out of 74 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| xtask/benchmark/benches/groups/compilation_stages.rs | Enable the new experiment in the concatenate-stage benchmark compiler configuration. |
| xtask/benchmark/benches/groups/bundle/util.rs | Enable the new experiment in the basic benchmark compiler builder experiments config. |
| website/docs/zh/config/experiments.mdx | Document experiments.fasterModuleConcatenation in Chinese docs (type/default/usage). |
| website/docs/en/config/experiments.mdx | Document experiments.fasterModuleConcatenation in English docs (type/default/usage). |
| tests/rspack-test/esmOutputCases/deconflict/destructured-export-name-collision/wrapper-b.js | New ESM output deconflict fixture for destructured exports (wrapper B). |
| tests/rspack-test/esmOutputCases/deconflict/destructured-export-name-collision/wrapper-a.js | New ESM output deconflict fixture for destructured exports (wrapper A). |
| tests/rspack-test/esmOutputCases/deconflict/destructured-export-name-collision/rspack.config.js | New test config enabling faster concatenation experiment for modern-module output. |
| tests/rspack-test/esmOutputCases/deconflict/destructured-export-name-collision/index.js | New ESM output deconflict test asserting bindings don’t collide. |
| tests/rspack-test/esmOutputCases/deconflict/destructured-export-name-collision/api-b.cjs | New CJS fixture module B for destructured export collision test. |
| tests/rspack-test/esmOutputCases/deconflict/destructured-export-name-collision/api-a.cjs | New CJS fixture module A for destructured export collision test. |
| tests/rspack-test/esmOutputCases/deconflict/destructured-export-name-collision/snapshots/runtimeModeSnapshot/esm.snap.txt | Add runtime-mode snapshot for the new destructured export collision ESM output case. |
| tests/rspack-test/esmOutputCases/deconflict/destructured-export-name-collision/snapshots/esm.snap.txt | Add webpack-mode snapshot for the new destructured export collision ESM output case. |
| tests/rspack-test/esmOutputCases/deconflict/destructured-direct-export/test.config.js | Add snapshot filtering to focus on the relevant wrapper region in output. |
| tests/rspack-test/esmOutputCases/deconflict/destructured-direct-export/rspack.config.js | New ESM output test config enabling faster concatenation experiment. |
| tests/rspack-test/esmOutputCases/deconflict/destructured-direct-export/index.js | New ESM output test for a destructured direct export path. |
| tests/rspack-test/esmOutputCases/deconflict/destructured-direct-export/snapshots/runtimeModeSnapshot/esm.snap.txt | Add runtime-mode snapshot for destructured direct export case. |
| tests/rspack-test/esmOutputCases/deconflict/destructured-direct-export/snapshots/esm.snap.txt | Add webpack-mode snapshot for destructured direct export case. |
| tests/rspack-test/esmOutputCases/deconflict/codegen-generated-identifiers/rspack.config.js | New ESM output test config enabling experiment for identifier deconflict coverage. |
| tests/rspack-test/esmOutputCases/deconflict/codegen-generated-identifiers/index.js | New test asserting deconflict of identifiers generated by dependency templates. |
| tests/rspack-test/esmOutputCases/deconflict/codegen-generated-identifiers/generated-default.js | New fixture module exporting a default value for snapshot assertions. |
| tests/rspack-test/esmOutputCases/deconflict/codegen-generated-identifiers/data.json | New JSON fixture to exercise placeholder/sentinel-related behavior. |
| tests/rspack-test/esmOutputCases/deconflict/codegen-generated-identifiers/snapshots/esm.snap.txt | Add snapshot for generated-identifier deconflict ESM output case. |
| tests/rspack-test/errorCases/faster-module-concatenation.js | Add error-case coverage verifying legacy parse behavior is preserved when experiment is off. |
| tests/rspack-test/defaultsCases/experiments/faster-module-concatenation.js | Add defaults-case coverage for experiments.fasterModuleConcatenation. |
| tests/rspack-test/defaultsCases/default/base.js | Update default options snapshot to include fasterModuleConcatenation: false. |
| tests/rspack-test/configCases/concatenate-modules/destructured-cjs-export-collision/wrapper-b.js | New concatenate-modules config case fixture (wrapper B) for destructured CJS export collision. |
| tests/rspack-test/configCases/concatenate-modules/destructured-cjs-export-collision/wrapper-a.js | New concatenate-modules config case fixture (wrapper A) for destructured CJS export collision. |
| tests/rspack-test/configCases/concatenate-modules/destructured-cjs-export-collision/rspack.config.js | New config case enabling experiment + concatenation to validate deconfliction. |
| tests/rspack-test/configCases/concatenate-modules/destructured-cjs-export-collision/package.json | Add sideEffects=false fixture metadata for the concatenation config case. |
| tests/rspack-test/configCases/concatenate-modules/destructured-cjs-export-collision/index.js | New test asserting concatenated destructured bindings are deconflicted in emitted output. |
| tests/rspack-test/configCases/concatenate-modules/destructured-cjs-export-collision/api-b.cjs | New CJS fixture B for concatenation deconflict config case. |
| tests/rspack-test/configCases/concatenate-modules/destructured-cjs-export-collision/api-a.cjs | New CJS fixture A for concatenation deconflict config case. |
| packages/rspack/src/config/types.ts | Expose experiments.fasterModuleConcatenation in TypeScript config types. |
| packages/rspack/src/config/normalization.ts | Add normalized experiments shape support for the new option. |
| packages/rspack/src/config/defaults.ts | Default experiments.fasterModuleConcatenation to false. |
| crates/rspack/tests/snapshots/defaults__default_options.snap | Update Rust defaults snapshot to include faster_module_concatenation: false. |
| crates/rspack/src/builder/mod.rs | Add builder plumbing for faster_module_concatenation. |
| crates/rspack_sources/src/source.rs | Extend AsAny to support Arc downcasting via into_any_arc. |
| crates/rspack_sources/src/replace_source.rs | Add in-place update helper for owned replacement contents. |
| crates/rspack_plugin_rstest/src/parser_plugin.rs | Gate ModulePathNameDependency range tracking on experiment flag. |
| crates/rspack_plugin_rstest/src/module_path_name_dependency.rs | Track replacement ranges and deconflict __dirname/__filename via concatenation scope when faster path is enabled. |
| crates/rspack_plugin_json/src/lib.rs | Register/deconflict JSON namespace export through generated top-level symbols under faster concatenation. |
| crates/rspack_plugin_javascript/src/visitors/semicolon.rs | Collect concatenation-scope identifier snapshot during semicolon insertion traversal when needed. |
| crates/rspack_plugin_javascript/src/visitors/dependency/parser/mod.rs | Expose compiler_options() to allow external plugins to query experiments. |
| crates/rspack_plugin_javascript/src/parser_plugin/esm_export_dependency_parser_plugin.rs | Preserve range info for default class export declarations under faster concatenation. |
| crates/rspack_plugin_javascript/src/parser_plugin/compatibility_plugin.rs | Adjust ConstDependency creation to carry concatenation identifier info under faster path. |
| crates/rspack_plugin_javascript/src/parser_and_generator/mod.rs | Store concatenation-scope snapshots and, for faster concatenated codegen, persist rendered init fragments separately. |
| crates/rspack_plugin_javascript/src/dependency/esm/provide_dependency.rs | Deconflict provided dependency identifiers via generated top-level symbols under faster concatenation. |
| crates/rspack_plugin_javascript/src/dependency/esm/mod.rs | Re-export NamedDeclarationInfo for faster concatenation default export handling. |
| crates/rspack_plugin_javascript/src/dependency/esm/import_meta_rsc_dependency.rs | Deconflict ImportMeta RSC binding via generated top-level symbols when faster path is active. |
| crates/rspack_plugin_javascript/src/dependency/esm/esm_import_specifier_dependency.rs | Deconflict import bindings / remove original ranges appropriately for faster concatenation. |
| crates/rspack_plugin_javascript/src/dependency/esm/esm_import_dependency.rs | Deconflict import vars and async-deps binding handling for faster concatenation. |
| crates/rspack_plugin_javascript/src/dependency/esm/esm_export_imported_specifier_dependency.rs | Handle faster concatenation by removing replaced ranges and emitting export references without reparsing. |
| crates/rspack_plugin_javascript/src/dependency/esm/esm_export_header_dependency.rs | Remove original export header range from snapshot tracking under faster concatenation. |
| crates/rspack_plugin_javascript/src/dependency/esm/esm_export_expression_dependency.rs | Add range-aware named default export declaration support and generated default export symbol handling. |
| crates/rspack_plugin_javascript/src/dependency/commonjs/require_resolve_header_dependency.rs | Remove original range tracking under faster concatenation when rewriting require.resolve header. |
| crates/rspack_plugin_javascript/src/dependency/commonjs/require_header_dependency.rs | Remove original range tracking under faster concatenation when rewriting require header. |
| crates/rspack_plugin_esm_library/src/render.rs | Render concatenated module sources with placeholder replacement without reparsing when experiment is enabled. |
| crates/rspack_plugin_esm_library/src/plugin.rs | Pass experiment flag into concatenation scope for ESM library rendering. |
| crates/rspack_plugin_esm_library/src/link.rs | Avoid codegen-time reparsing by populating concatenation info from snapshots when faster path is enabled. |
| crates/rspack_plugin_css/src/parser_and_generator/generator.rs | Deconflict CSS export identifiers using generated top-level symbols under faster concatenation. |
| crates/rspack_plugin_asset/src/lib.rs | Deconflict asset namespace export binding via generated top-level symbols under faster concatenation. |
| crates/rspack_core/src/utils/concatenation_scope.rs | Add faster-path helpers: range removals, used-name tracking, generated symbol placeholders, and module reference placeholder tracking. |
| crates/rspack_core/src/utils/concatenated_module_visitor.rs | Introduce cacheable concatenation scope snapshot structures for range-only identifier tracking. |
| crates/rspack_core/src/options/experiments/mod.rs | Add faster_module_concatenation to core experiments options. |
| crates/rspack_core/src/module.rs | Persist optional concatenation_scope_snapshot in BuildInfo. |
| crates/rspack_core/src/init_fragment.rs | Add string-rendering helper for init fragments + binding support for await-deps fragments. |
| crates/rspack_core/src/external_module.rs | Deconflict namespace export binding for externals under faster concatenation. |
| crates/rspack_core/src/dependency/const_dependency.rs | Carry optional concatenation-scope identifier info and integrate placeholder/used-name tracking under faster concatenation. |
| crates/rspack_core/src/dependency/cached_const_dependency.rs | Deconflict cached const identifiers and ensure decl symbols are tracked under faster concatenation. |
| crates/rspack_core/src/concatenated_module.rs | Populate concatenation info from snapshots and render concatenated output with placeholder replacement instead of reparsing. |
| crates/rspack_core/src/artifacts/code_generation_results.rs | Introduce CodeGenerationDataRenderedInitFragments to persist init fragments as strings for the faster path. |
| crates/rspack_binding_api/src/raw_options/raw_experiments/mod.rs | Thread the new experiment flag through Rust binding raw options into core Experiments. |
| crates/node_binding/napi-binding.d.ts | Expose fasterModuleConcatenation in the Node binding type declarations. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Contributor
Contributor
Contributor
Author
|
@codspeedbot explain why this PR is faster |
Contributor
Why this PR is fasterI compared the head run ( The base path re-parsed generated code during code generationOn
So the pipeline was: render the source, then re-lex, re-parse, re-resolve scopes, and re-walk the AST just to rebuild name-analysis state for deconfliction. The head path resolves bindings from a snapshot insteadWhen
In other words, the collection work is fused into a visitor that already runs at make time, range-only records are stored, and generated bindings are resolved lazily from code-generation metadata — so the expensive lex→parse→resolve→walk round-trip on generated sources no longer happens on the code-generation hot path. That eliminated block is essentially the full delta between the two runs. A note on the reported numbersCodSpeed flagged "Different runtime environments detected" on this comparison — base and head ran on different CPUs (EPYC 9V74 → 7763, with AVX-512 dropped). In Simulation mode that can shift the estimate a bit on identical source, so treat the exact percentages (e.g. the Summary
Was this response helpful? React with 👍 or 👎 to help us improve CodSpeedBot. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
experiments.fasterModuleConcatenationswitch, defaulting tofalseThe earlier approach in #14033 added extra AST traversal and persisted high-cardinality identifier/reference maps for every eligible ESM module. That expanded make, allocation, serialization, and cache-restore work while still rebuilding name-analysis state during code generation. This version fuses collection into an existing visitor, stores range-only records, and resolves generated bindings lazily from code-generation metadata.
The CodSpeed benchmarks that exercise module concatenation explicitly enable the experiment, so the performance report compares the new path against the legacy implementation. The previous always-on head reported a 13.93% overall improvement and a 14.80% improvement in
rust@concatenate_module_code_generation; the experiment-gated head will be evaluated by the new CI run.Related links
Checklist