feat: compress native npm bindings#12000
Conversation
![kodiakhq[bot]](https://avatars.githubusercontent.com/in/29196?s=60&v=4){kind=small}
🦋 Changeset detectedLatest commit: 10f40f6 The changes in this PR will be included in the next version bump. Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
|
Review the following changes in direct dependencies. Learn more about Socket for GitHub.
|
|
Warning Review the following alerts detected in dependencies. According to your organization's Security Policy, it is recommended to resolve "Warn" alerts. Learn more about Socket for GitHub.
|
kdy1
left a comment
kdy1
left a comment
There was a problem hiding this comment.
This is an experiment PR to see the size impact
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 5d64b0e09b
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Merging this PR will not alter performance
Comparing 
Footnotes
|
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 10f40f601f
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| } | ||
| } | ||
|
|
||
| write_cache_addon(&cache_path, &raw)?; |
There was a problem hiding this comment.
When HOME/XDG_CACHE_HOME/LOCALAPPDATA is set but points at a read-only location, this propagated cache write error makes require('@swc/...') fail instead of trying the temp fallback or another writable path; on Unix this can even fail the first load after try_self_replace has already succeeded because the cache write is still mandatory. This regresses containers/serverless images with non-writable home directories, so the loader should fall back to a writable temp/cache path or load from a temporary file before returning an error.
Useful? React with 👍 / 👎.
| ], | ||
| "engines": { | ||
| "node": ">=10" | ||
| "node": ">=20" |
There was a problem hiding this comment.
This engine range still allows Node 20.0–20.2, but the new loader is built with napi-sys's napi9 feature and calls node_api_get_module_file_name, which is gated on N-API 9. Those early Node 20 releases only expose N-API 8, so npm can install this package and then the native loader fails to resolve/execute the symbol at require time; please raise the range to the first Node 20 release with N-API 9 or avoid that API.
Useful? React with 👍 / 👎.
| let raw_addon = find_single_node_addon(&raw_dir)?; | ||
| let raw = fs::read(&raw_addon) | ||
| .with_context(|| format!("failed to read staged raw addon {}", raw_addon.display()))?; | ||
| let pressed = pack_pressed_data(&raw, 19)?; |
There was a problem hiding this comment.
In the macOS x64 publish job I checked .github/workflows/publish-npm-package.yml, where strip -x *.node runs after pnpm build; with this line the raw addon has already been read into the compressed payload, so that later strip only trims the small loader and the shipped payload remains unstripped. This regresses the release package/download size for that target, so the raw .node needs to be stripped before pack_pressed_data embeds it.
Useful? React with 👍 / 👎.
| ) | ||
| })?; | ||
| } | ||
| let outcome = compress_bytes(cache_path, raw, &Gate::any()).map_err(|err| { |
There was a problem hiding this comment.
When multiple Node processes first load the same binding with an empty shared cache, they all decode and write the deterministic cache_path directly; one process can finish its write and then race with another process truncating/replacing the same .node before load_and_register, producing an intermittent invalid native module load. Please write to a unique temp path and rename into the cache only after the file is complete.
Useful? React with 👍 / 👎.
|
9.2M (9557416 bytes) |
1 participant
Description:
Apply compressed native binding loading to the SWC native npm packages. Native
.nodebuilds now go through a shared wrapper that first builds the raw napi addon, packs it as zstd pressed-data with SHA-512 integrity, and then builds a small Rust loader as the shipped.nodeartifact.The loader verifies and decompresses the embedded payload on first load, then materializes the raw addon through OS transparent compression when available or a per-user cache fallback otherwise. This keeps the public JS require path and exported N-API surface unchanged while reducing native package download/install size.
This also raises all native SWC npm package
engines.nodeentries, including platform packages, to Node 20 or newer, and updates native CI jobs to Node 20.Validation performed locally:
git submodule update --init --recursivecargo fmt --allcargo clippy --all --all-targets -- -D warningscargo test -p swc_compressed_binding_toolcargo test -p binding_compressed_node_loadercargo test -p swc-releasercd packages/core && pnpm build:dev && SWC_NATIVE_BINDING_DISABLE_SELF_REPLACE=1 SWC_NATIVE_BINDING_DEBUG=1 pnpm testcd packages/html && pnpm build:dev && SWC_NATIVE_BINDING_DISABLE_SELF_REPLACE=1 SWC_NATIVE_BINDING_DEBUG=1 pnpm testcd packages/html && SWC_NATIVE_BINDING_DISABLE_SELF_REPLACE=1 SWC_NATIVE_BINDING_DEBUG=1 node -e "const html = require('./'); const out = html.minifySync('<div> hi </div>', {}); console.log(out.code);"cd packages/minifier && pnpm build:dev && SWC_NATIVE_BINDING_DISABLE_SELF_REPLACE=1 SWC_NATIVE_BINDING_DEBUG=1 pnpm testcd packages/minifier && SWC_NATIVE_BINDING_CACHE_DIR=$(mktemp -d) SWC_NATIVE_BINDING_DISABLE_SELF_REPLACE=1 SWC_NATIVE_BINDING_DEBUG=1 node -e "require('./').minifySync('function f(){ return 1 }', {})"cd packages/react-compiler && pnpm build:dev && SWC_NATIVE_BINDING_DISABLE_SELF_REPLACE=1 SWC_NATIVE_BINDING_DEBUG=1 pnpm testBREAKING CHANGE:
Native SWC npm packages now require Node.js 20 or newer.
Related issue (if exists):
N/A