Skip to content

perf: cache module concatenation analysis across roots#14832

Open
LingyuCoder wants to merge 1 commit into
seal/module-concat-benchmarksfrom
seal/module-concat-runtime-cache
Open

perf: cache module concatenation analysis across roots#14832
LingyuCoder wants to merge 1 commit into
seal/module-concat-benchmarksfrom
seal/module-concat-runtime-cache

Conversation

@LingyuCoder

@LingyuCoder LingyuCoder commented Jul 17, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Summary

  • Cache runtime-dependent incoming-module analysis by module and runtime across root searches.
  • Keep root-specific eligibility, chunk-membership, and ESM checks outside the shared cache.
  • Add multi-runtime regression coverage to ensure cache entries remain isolated by runtime.

Related links

Checklist

  • Tests updated.
  • Documentation not required.

@LingyuCoder LingyuCoder mentioned this pull request Jul 17, 2026
Open
2 tasks
@LingyuCoder LingyuCoder changed the title perf(module-concat): cache incoming analysis across roots perf: cache module concatenation analysis across roots Jul 17, 2026
@LingyuCoder LingyuCoder mentioned this pull request Jul 17, 2026
Open
2 tasks
@github-actions

github-actions Bot commented Jul 17, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

📝 Benchmark detail: Open

Name Base (25274d8) Current Change
10000_big_production-mode_disable-minimize + exec 12.1 s ± 334 ms 12.1 s ± 313 ms -0.19 %
10000_development-mode + exec 895 ms ± 19 ms 887 ms ± 36 ms -0.92 %
10000_development-mode_hmr + stats 162 ms ± 18 ms 155 ms ± 3.3 ms -4.35 %
10000_development-mode_noop-loader + exec 1.78 s ± 135 ms 1.76 s ± 133 ms -1.11 %
10000_production-mode + exec 1.02 s ± 32 ms 1.02 s ± 16 ms -0.76 %
10000_production-mode_persistent-cold + exec 1.21 s ± 38 ms 1.19 s ± 32 ms -1.31 %
10000_production-mode_persistent-hot + exec 634 ms ± 12 ms 632 ms ± 10 ms -0.41 %
10000_production-mode_source-map + exec 1.21 s ± 97 ms 1.21 s ± 129 ms -0.41 %
arco-pro_development-mode + exec 1.25 s ± 40 ms 1.28 s ± 75 ms +2.54 %
arco-pro_development-mode_hmr + stats 33 ms ± 1.3 ms 33 ms ± 0.39 ms -1.04 %
arco-pro_production-mode + exec 2.34 s ± 154 ms 2.32 s ± 161 ms -0.93 %
arco-pro_production-mode_generate-package-json-webpack-plugin + exec 2.42 s ± 84 ms 2.4 s ± 76 ms -0.52 %
arco-pro_production-mode_persistent-cold + exec 2.4 s ± 129 ms 2.4 s ± 87 ms -0.27 %
arco-pro_production-mode_persistent-hot + exec 344 ms ± 9 ms 348 ms ± 7.9 ms +1.22 %
arco-pro_production-mode_source-map + exec 2.84 s ± 68 ms 2.8 s ± 139 ms -1.52 %
arco-pro_production-mode_source-map_persistent-cold + exec 2.95 s ± 129 ms 2.92 s ± 120 ms -0.88 %
arco-pro_production-mode_source-map_persistent-hot + exec 477 ms ± 7.1 ms 474 ms ± 11 ms -0.71 %
arco-pro_production-mode_traverse-chunk-modules + exec 2.37 s ± 143 ms 2.38 s ± 115 ms +0.50 %
bundled-threejs_development-mode + exec 181 ms ± 2.9 ms 180 ms ± 2 ms -0.44 %
bundled-threejs_production-mode + exec 208 ms ± 5.7 ms 208 ms ± 5.3 ms +0.10 %
large-dyn-imports_development-mode + exec 1.17 s ± 31 ms 1.16 s ± 23 ms -0.58 %
large-dyn-imports_production-mode + exec 1.27 s ± 30 ms 1.27 s ± 72 ms +0.11 %
threejs_development-mode_10x + exec 775 ms ± 13 ms 784 ms ± 14 ms +1.16 %
threejs_development-mode_10x_hmr + stats 112 ms ± 2.7 ms 107 ms ± 5.4 ms -4.74 %
threejs_production-mode_10x + exec 2.86 s ± 136 ms 2.8 s ± 40 ms -1.97 %
threejs_production-mode_10x_persistent-cold + exec 2.95 s ± 26 ms 2.93 s ± 25 ms -0.64 %
threejs_production-mode_10x_persistent-hot + exec 380 ms ± 7.9 ms 374 ms ± 4.7 ms -1.53 %
threejs_production-mode_10x_source-map + exec 3.66 s ± 179 ms 3.59 s ± 25 ms -1.99 %
10000_big_production-mode_disable-minimize + rss memory 2179 MiB ± 83.1 MiB 2194 MiB ± 28 MiB +0.71 %
10000_development-mode + rss memory 538 MiB ± 14 MiB 539 MiB ± 14.8 MiB +0.33 %
10000_development-mode_hmr + rss memory 788 MiB ± 40.7 MiB 789 MiB ± 35.3 MiB +0.20 %
10000_development-mode_noop-loader + rss memory 805 MiB ± 11.9 MiB 807 MiB ± 7.24 MiB +0.20 %
10000_production-mode + rss memory 466 MiB ± 9.03 MiB 464 MiB ± 5.4 MiB -0.55 %
10000_production-mode_persistent-cold + rss memory 662 MiB ± 11.1 MiB 662 MiB ± 12.8 MiB -0.06 %
10000_production-mode_persistent-hot + rss memory 649 MiB ± 33.8 MiB 646 MiB ± 29.7 MiB -0.39 %
10000_production-mode_source-map + rss memory 483 MiB ± 32.9 MiB 479 MiB ± 27.3 MiB -0.79 %
arco-pro_development-mode + rss memory 399 MiB ± 2.04 MiB 398 MiB ± 4.79 MiB -0.21 %
arco-pro_development-mode_hmr + rss memory 422 MiB ± 8.17 MiB 421 MiB ± 3.87 MiB -0.20 %
arco-pro_production-mode + rss memory 574 MiB ± 12.9 MiB 576 MiB ± 9.88 MiB +0.27 %
arco-pro_production-mode_generate-package-json-webpack-plugin + rss memory 588 MiB ± 8.89 MiB 590 MiB ± 13.9 MiB +0.36 %
arco-pro_production-mode_persistent-cold + rss memory 660 MiB ± 26.3 MiB 660 MiB ± 19.5 MiB +0.09 %
arco-pro_production-mode_persistent-hot + rss memory 293 MiB ± 9.15 MiB 294 MiB ± 4.61 MiB +0.27 %
arco-pro_production-mode_source-map + rss memory 682 MiB ± 11.2 MiB 679 MiB ± 22.1 MiB -0.42 %
arco-pro_production-mode_source-map_persistent-cold + rss memory 812 MiB ± 37.6 MiB 811 MiB ± 14 MiB -0.08 %
arco-pro_production-mode_source-map_persistent-hot + rss memory 413 MiB ± 12.3 MiB 417 MiB ± 19 MiB +0.98 %
arco-pro_production-mode_traverse-chunk-modules + rss memory 579 MiB ± 11.1 MiB 579 MiB ± 9.34 MiB +0.06 %
bundled-threejs_development-mode + rss memory 126 MiB ± 3.29 MiB 128 MiB ± 1015 KiB +1.69 %
bundled-threejs_production-mode + rss memory 137 MiB ± 2.18 MiB 140 MiB ± 5.41 MiB +2.10 %
large-dyn-imports_development-mode + rss memory 553 MiB ± 20.4 MiB 556 MiB ± 6.9 MiB +0.55 %
large-dyn-imports_production-mode + rss memory 406 MiB ± 1.63 MiB 406 MiB ± 5.69 MiB +0.20 %
threejs_development-mode_10x + rss memory 432 MiB ± 9.74 MiB 437 MiB ± 4.37 MiB +1.28 %
threejs_development-mode_10x_hmr + rss memory 592 MiB ± 10.9 MiB 603 MiB ± 16.2 MiB +1.83 %
threejs_production-mode_10x + rss memory 615 MiB ± 22.8 MiB 619 MiB ± 17.9 MiB +0.66 %
threejs_production-mode_10x_persistent-cold + rss memory 576 MiB ± 12 MiB 575 MiB ± 5.56 MiB -0.27 %
threejs_production-mode_10x_persistent-hot + rss memory 378 MiB ± 7.36 MiB 371 MiB ± 9.5 MiB -1.79 %
threejs_production-mode_10x_source-map + rss memory 738 MiB ± 26.3 MiB 742 MiB ± 31.4 MiB +0.57 %

Base persistent cache hit rate: 👍

Current persistent cache hit rate: 👍

@github-actions

github-actions Bot commented Jul 17, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

📦 Binary Size-limit

Comparing eff6ba1 to Merge 6ab226258457294f7647575ec7b95199c6730210 into a0e983a95e074ea3ff0cf14f4112c12e8d38ea92 by harpsealjs

⏳ The base commit triggered a linux binding build, but its binary size data has not been generated yet, so the size comparison is skipped.

Please re-run this workflow once the ecosystem-benchmark data for that commit is published.

Warning

Reference only — not the real baseline. The base commit's data isn't ready yet, so this compares against the nearest earlier commit that has data (a0e983a) for a rough estimate:

❌ Size increased by 12.00KB from 66.65MB to 66.66MB (⬆️0.02%)

@github-actions

github-actions Bot commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

Rsdoctor Bundle Diff Analysis

Found 5 projects in monorepo, 5 projects with changes.

📊 Quick Summary
Project Total Size Gzip Size Change Gzip Change
popular-libs 1.7 MB 552.1 KB - -
react-10k 5.6 MB 1.3 MB - -
react-1k 823.1 KB 217.0 KB - -
react-5k 2.7 MB 663.8 KB - -
ui-components 4.9 MB 1.4 MB - -
📋 Detailed Reports (Click to expand)

📁 popular-libs

Path: ../build-tools-performance/cases/popular-libs/dist/rsdoctor-data.json

⚠️ No baseline data found - Unable to perform comparison analysis

Metric Current Baseline Change
📊 Total Size 1.7 MB - -
🗜️ Gzip Size 552.1 KB - -
📄 JavaScript 1.7 MB - -
🎨 CSS 0 B - -
🌐 HTML 289.0 B - -
📁 Other Assets 0 B - -

📁 react-10k

Path: ../build-tools-performance/cases/react-10k/dist/rsdoctor-data.json

⚠️ No baseline data found - Unable to perform comparison analysis

Metric Current Baseline Change
📊 Total Size 5.6 MB - -
🗜️ Gzip Size 1.3 MB - -
📄 JavaScript 5.6 MB - -
🎨 CSS 21.0 B - -
🌐 HTML 328.0 B - -
📁 Other Assets 0 B - -

📁 react-1k

Path: ../build-tools-performance/cases/react-1k/dist/rsdoctor-data.json

⚠️ No baseline data found - Unable to perform comparison analysis

Metric Current Baseline Change
📊 Total Size 823.1 KB - -
🗜️ Gzip Size 217.0 KB - -
📄 JavaScript 822.7 KB - -
🎨 CSS 0 B - -
🌐 HTML 328.0 B - -
📁 Other Assets 0 B - -

📁 react-5k

Path: ../build-tools-performance/cases/react-5k/dist/rsdoctor-data.json

⚠️ No baseline data found - Unable to perform comparison analysis

Metric Current Baseline Change
📊 Total Size 2.7 MB - -
🗜️ Gzip Size 663.8 KB - -
📄 JavaScript 2.7 MB - -
🎨 CSS 21.0 B - -
🌐 HTML 328.0 B - -
📁 Other Assets 0 B - -

📁 ui-components

Path: ../build-tools-performance/cases/ui-components/dist/rsdoctor-data.json

⚠️ No baseline data found - Unable to perform comparison analysis

Metric Current Baseline Change
📊 Total Size 4.9 MB - -
🗜️ Gzip Size 1.4 MB - -
📄 JavaScript 4.8 MB - -
🎨 CSS 112.1 KB - -
🌐 HTML 328.0 B - -
📁 Other Assets 0 B - -

Generated by Rsdoctor GitHub Action

@codspeed-hq

codspeed-hq Bot commented Jul 17, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Merging this PR will improve performance by 19.76%

⚡ 4 improved benchmarks
✅ 42 untouched benchmarks
⏩ 47 skipped benchmarks1

Performance Changes

Mode Benchmark BASE HEAD Efficiency
⚡ Simulation rust@create_concatenate_module_unsupported_syntax 30.3 ms 22.4 ms +35.31%
⚡ Simulation rust@create_concatenate_module_shared_roots 30 ms 22.2 ms +35.06%
⚡ Simulation rust@create_concatenate_module_bailouts 107.3 ms 99.3 ms +7.99%
⚡ Simulation rust@create_concatenate_module 14.4 ms 13.8 ms +4.25%

Tip

Curious why this is faster? Comment @codspeedbot explain why this is faster on this PR, or directly use the CodSpeed MCP with your agent.

Comparing seal/module-concat-runtime-cache (eff6ba1) with seal/module-concat-benchmarks (e48b090)2

Open in CodSpeed

Footnotes

  1. 47 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

  2. No successful run was found on seal/module-concat-benchmarks (6ab2262) during the generation of this report, so 9a7fcc8 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

@LingyuCoder

LingyuCoder commented Jul 21, 2026

Copy link
Copy Markdown
Contributor Author

@codspeedbot explain why this PR is faster

@codspeed-hq

codspeed-hq Bot commented Jul 21, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

@codspeedbot explain why this PR is faster

Here's why this PR speeds up the create_concatenate_module* benchmarks.

What got faster

Benchmark Base Head Change
create_concatenate_module_shared_roots 30.1 ms 22.3 ms +35.17%
create_concatenate_module_unsupported_syntax 30.2 ms 22.4 ms +35.12%
create_concatenate_module_bailouts 107.3 ms 99.2 ms +8.18%
create_concatenate_module 14.4 ms 13.8 ms +4.09%

Root cause

ModuleConcatenationPlugin builds concatenation groups by picking each relevant module as a root and walking its incoming connections via try_to_add to decide which modules can be merged in. Before this PR, the runtime-dependent incoming analysis for a module was recomputed from scratch on every root search that touched it. When many roots share overlapping module neighborhoods, the same module's incoming edges get analyzed over and over.

This PR introduces a RuntimeIdentifierCache<IncomingModulesCacheEntry> (incoming_modules_cache) that is allocated once per optimize_chunk_modules pass (outside the per-root loop, module_concatenation_plugin.rs:1084) and threaded into every try_to_add call (:1269). The first time a (module, runtime) pair is analyzed, the result — either the list of active incoming modules or a RuntimeDependent bailout — is stored (:730, :742); subsequent root searches hit the cache and return the memoized result immediately (:557–:566, tracked as statistics.cache_hit). The companion imports_cache memoizes get_imports the same way.

The key design detail that makes the cache correct is the split between what is cached and what stays per-root:

  • Cached (runtime-dependent, root-independent): the expensive is_target_active / filter_runtime connection-activity analysis, keyed by both module and runtime (no_runtime_map + a per-RuntimeSpec runtime_map), so a cache entry is never reused across a different runtime.
  • Not cached (root-specific): chunk-membership checks (root_chunks.is_subset(...), :755) and the ESM/is_esm checks are still evaluated per root, since those genuinely depend on which root you started from.

Why the improvement is uneven

  • shared_roots and unsupported_syntax (~+35%) benefit the most because they exercise exactly the pattern the cache targets: many roots re-examining overlapping incoming modules, so the redundant activity analysis dominated the base runtime.
  • bailouts (+8%) and the baseline create_concatenate_module (+4%) gain less because there's less cross-root overlap to amortize.

The isolation-by-runtime keying is also covered by the new multi-runtime regression test mentioned in the PR description, which guards against a cached entry leaking across runtimes.

About the flagged regression

The rust@create_chunk_ids "regression" (3 ms → 3.1 ms, -2.29%) is unrelated to this change — that path lives in rspack_ids/chunk-graph code, not module concatenation. It's one of the known short, noise-prone stage benchmarks (~3 ms), and CodSpeed flagged a "Different runtime environments detected" warning on this comparison, so a sub-3% swing there is jitter rather than a real regression from this PR.

Summary
  • Warming up the performance wizard...
  • Investigating the speedup — Confirmed the source of the module concatenation speedup from the code and benchmark report.
  • Handling comment — Comment handling finished successfully

Was this response helpful? React with 👍 or 👎 to help us improve CodSpeedBot.

@LingyuCoder
LingyuCoder marked this pull request as ready for review July 21, 2026 06:34
Copilot AI review requested due to automatic review settings July 21, 2026 06:34
Copilot AI reviewed Jul 21, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@LingyuCoder
LingyuCoder requested a review from JSerFeng July 21, 2026 07:39
@LingyuCoder @ScriptedAlchemy
perf(module-concat): cache incoming analysis across roots
eff6ba1 
Cache runtime-dependent incoming module analysis across root searches while keeping root-specific chunk and ESM checks outside the cache. Add runtime-isolation coverage for multi-runtime builds.

Co-authored-by: ScriptedAlchemy <zackary.l.jackson@gmail.com>
@LingyuCoder
LingyuCoder force-pushed the seal/module-concat-benchmarks branch from e48b090 to 6ab2262 Compare July 21, 2026 07:44
@LingyuCoder
LingyuCoder force-pushed the seal/module-concat-runtime-cache branch from 9924a3e to eff6ba1 Compare July 21, 2026 07:44
@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Jul 21, 2026 •
edited
Loading

Copy link
Copy Markdown

Deploying rspack with  Cloudflare Pages  Cloudflare Pages

Latest commit: eff6ba1
Status: ✅  Deploy successful!
Preview URL: https://c7ee7e6f.rspack-v2.pages.dev
Branch Preview URL: https://seal-module-concat-runtime-c.rspack-v2.pages.dev

View logs

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot Copilot left review comments
@JSerFeng JSerFeng JSerFeng approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@LingyuCoder @JSerFeng