Summary
On Linux, experiments.nativeWatcher registers every ignored descendant of a
recursive context dependency before applying watchOptions.ignored. This
adds avoidable lazy-startup work, consumes inotify descriptors, and can reach
max_user_watches on large repositories. Watchpack prunes the same tree and
keeps the descriptor count flat.
Reproduction and measurements
Create an entry plus eight live nested directories and an ignored tree, add
their common parent to compilation.contextDependencies, enable
experiments.nativeWatcher, and set watchOptions.ignored to
'**/ignored/**'. Count inotify wd: lines in /proc/<pid>/fdinfo/* after
the first watch callback, then write one ignored file and one live nested file.
| ignored top-level dirs |
physical ignored dirs |
native watches |
native ready |
Watchpack watches |
Watchpack ready |
| 0 |
1 |
23 |
~0.20 s |
23 |
~0.29 s |
| 100 |
201 |
223 |
~0.22 s |
23 |
~0.29 s |
| 1,000 |
2,001 |
2,023 |
~0.34 s |
23 |
~0.30 s |
| 5,000 |
10,001 |
10,023 |
~1.25 s |
23 |
~0.32 s |
The native no-ignore run has the same registration counts. Ignored writes do
not rebuild and live nested writes do, so event filtering is correct; the
problem is specifically registration/traversal cost.
Cause and suggested direction
The inotify backend recursively walks/registers descendants for a recursive
watch and also recursively registers newly created or moved-in directories.
Rspack currently filters in the event trigger, after that work is complete.
A single non-recursive context watch is not sufficient because it would miss
changes in pre-existing children.
The maintainable fix is an ignored-aware recursive registration path (ideally
a prune predicate in the watcher backend) that applies the same filter during
both the initial walk and later create/rename registration. An in-Rspack Linux
registration actor could alternatively register retained directories
non-recursively, process directory create/rename/remove events without calling
synchronous watch() from the notify callback, and preserve symlink,
watch-limit, and close/rebuild semantics. This should have focused tests for
ignored/live siblings, nested creation, rename in/out, symlinks, and races.
This was found while validating the consolidated HMR/lazy-compilation work in
#14772; it is intentionally separate from the correctness fixes in that PR.
Summary
On Linux,
experiments.nativeWatcherregisters every ignored descendant of arecursive context dependency before applying
watchOptions.ignored. Thisadds avoidable lazy-startup work, consumes inotify descriptors, and can reach
max_user_watcheson large repositories. Watchpack prunes the same tree andkeeps the descriptor count flat.
Reproduction and measurements
Create an entry plus eight live nested directories and an ignored tree, add
their common parent to
compilation.contextDependencies, enableexperiments.nativeWatcher, and setwatchOptions.ignoredto'**/ignored/**'. Countinotify wd:lines in/proc/<pid>/fdinfo/*afterthe first watch callback, then write one ignored file and one live nested file.
The native no-ignore run has the same registration counts. Ignored writes do
not rebuild and live nested writes do, so event filtering is correct; the
problem is specifically registration/traversal cost.
Cause and suggested direction
The inotify backend recursively walks/registers descendants for a recursive
watch and also recursively registers newly created or moved-in directories.
Rspack currently filters in the event trigger, after that work is complete.
A single non-recursive context watch is not sufficient because it would miss
changes in pre-existing children.
The maintainable fix is an ignored-aware recursive registration path (ideally
a prune predicate in the watcher backend) that applies the same filter during
both the initial walk and later create/rename registration. An in-Rspack Linux
registration actor could alternatively register retained directories
non-recursively, process directory create/rename/remove events without calling
synchronous
watch()from the notify callback, and preserve symlink,watch-limit, and close/rebuild semantics. This should have focused tests for
ignored/live siblings, nested creation, rename in/out, symlinks, and races.
This was found while validating the consolidated HMR/lazy-compilation work in
#14772; it is intentionally separate from the correctness fixes in that PR.