Skip to content

[Perf]: nativeWatcher registers ignored descendants for recursive context dependencies #14789

Description

@matthewdavis-oai

Summary

On Linux, experiments.nativeWatcher registers every ignored descendant of a
recursive context dependency before applying watchOptions.ignored. This
adds avoidable lazy-startup work, consumes inotify descriptors, and can reach
max_user_watches on large repositories. Watchpack prunes the same tree and
keeps the descriptor count flat.

Reproduction and measurements

Create an entry plus eight live nested directories and an ignored tree, add
their common parent to compilation.contextDependencies, enable
experiments.nativeWatcher, and set watchOptions.ignored to
'**/ignored/**'. Count inotify wd: lines in /proc/<pid>/fdinfo/* after
the first watch callback, then write one ignored file and one live nested file.

ignored top-level dirs physical ignored dirs native watches native ready Watchpack watches Watchpack ready
0 1 23 ~0.20 s 23 ~0.29 s
100 201 223 ~0.22 s 23 ~0.29 s
1,000 2,001 2,023 ~0.34 s 23 ~0.30 s
5,000 10,001 10,023 ~1.25 s 23 ~0.32 s

The native no-ignore run has the same registration counts. Ignored writes do
not rebuild and live nested writes do, so event filtering is correct; the
problem is specifically registration/traversal cost.

Cause and suggested direction

The inotify backend recursively walks/registers descendants for a recursive
watch and also recursively registers newly created or moved-in directories.
Rspack currently filters in the event trigger, after that work is complete.
A single non-recursive context watch is not sufficient because it would miss
changes in pre-existing children.

The maintainable fix is an ignored-aware recursive registration path (ideally
a prune predicate in the watcher backend) that applies the same filter during
both the initial walk and later create/rename registration. An in-Rspack Linux
registration actor could alternatively register retained directories
non-recursively, process directory create/rename/remove events without calling
synchronous watch() from the notify callback, and preserve symlink,
watch-limit, and close/rebuild semantics. This should have focused tests for
ignored/live siblings, nested creation, rename in/out, symlinks, and races.

This was found while validating the consolidated HMR/lazy-compilation work in
#14772; it is intentionally separate from the correctness fixes in that PR.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions