Skip to content

fix(core): disable output tracking watcher on Windows - #36566

Draft
sdjayna wants to merge 2 commits into
nrwl:masterfrom
sdjayna:fix/windows-watcher-handles
Draft

fix(core): disable output tracking watcher on Windows#36566
sdjayna wants to merge 2 commits into
nrwl:masterfrom
sdjayna:fix/windows-watcher-handles

Conversation

@sdjayna

@sdjayna sdjayna commented Aug 4, 2026

Copy link
Copy Markdown

Current Behavior

The Nx daemon starts two workspace-wide watchers: a source watcher and an output-tracking watcher. On Windows, both enumerate eligible directories and register them non-recursively. notify retains a directory HANDLE and completion-semaphore HANDLE for every registration.

For normal source directories, this duplicates registration cost: both watchers allocate a directory HANDLE and completion semaphore. Git-ignored directories are excluded by the source watcher but still registered by the broad output watcher.

The public reproduction uses Git-ignored directories specifically to isolate the output watcher's two-HANDLE slope. The PR removes that watcher across the workspace; it is not a Git-ignore-specific fix.

Expected Behavior

Windows should not pay a second workspace-wide per-directory HANDLE and memory cost for an output-tracking optimization. Cache correctness must be preserved through conservative misses, and source watching must remain unchanged.

After the fix:

  • normal source directories keep one source-watcher registration instead of two watcher registrations;
  • Git-ignored directories no longer receive output-watcher registrations; and
  • hard-excluded directories remain unwatched.

Before / After Acceptance

The before values are measured on stock Nx 23.1.1. The after values are acceptance targets, not patched results yet.

Metric from 0 to 131,072 Git-ignored directories Before: stock Nx, measured After: patched acceptance Status
Additional daemon HANDLEs +262,146; slope 2.000008/directory ≤64 total increase Awaiting dedicated PR release on native Windows
Additional private memory +2,214.7 MB; slope ~0.0169 MB/directory No linear directory-count growth Awaiting patched measurement
Graph command 2,664ms → 26,146ms; slope 0.180ms/directory No directory-count growth from output tracking Awaiting patched measurement
Daemon release after reset 4/4 exited 4/4 exit Existing reset behavior preserved
Output-cache correctness Output watcher invalidates recorded hashes Conservative cache miss (false) 6/6 focused tests pass
Source watching Existing behavior Unchanged No native watcher code modified

Raw public measurements:

Generated directories Daemon HANDLEs Private memory (MB) Working set (MB) Graph command
0 288 96.6 91.1 2,664ms
8,192 16,674 236.3 230.4 3,912ms
32,768 65,826 644.4 639.7 8,110ms
131,072 262,434 2,311.2 2,303.7 26,146ms

The regressions are nearly perfectly linear: HANDLE R² ≈ 1.0, private-memory R² 0.99999, and graph-command R² 0.99969. The fixture is Git-ignored, so this primarily isolates the broad output watcher. The memory slope is consistent with notify's 16 KB per-watch buffer plus overhead, but is not presented as byte-level attribution.

A separate large-workspace field observation reached 161,876 handles and approximately 2.48 GB. It motivated the investigation; the public 131,072-directory run independently exceeds its handle count and reaches the same multi-gigabyte process range.

What This PR Changes

  • Does not start the optional output-tracking watcher on Windows.
  • Calls Nx’s existing disableOutputsTracking() fallback.
  • Makes output-hash comparison return conservative misses rather than risk stale hits.
  • Leaves source watching and project-graph invalidation unchanged.
  • Adds a focused regression proving disabled output tracking returns false even for a previously recorded matching hash.

Testing

Completed locally:

Required with a dedicated PR release:

  • Re-run the same native Windows HANDLE measurement.
  • Verify the 0-to-131,072 ignored-directory increase no longer adds handles linearly.
  • Compare local-cache replay time and output-copy behavior before/after.
  • Verify source watcher behavior is unchanged in the public repro and a large workspace.

Risk and Mitigation

The trade-off is performance, not correctness: Windows may perform additional output restoration/copy work because output tracking returns conservative misses. The source watcher remains active, and no file-change event used for project-graph correctness is removed.

If the cache-copy cost is material, a future Windows output tracker should watch only known output roots or use a bounded backend. It should not return to a second workspace-wide per-directory watcher.

Related Issue(s)

Fixes #36563

This draft requests a dedicated PR release for measured Windows HANDLE, memory and cache-replay before/after results.

@netlify

netlify Bot commented Aug 4, 2026

Copy link
Copy Markdown

👷 Deploy request for nx-docs pending review.

Visit the deploys page to approve it

Name Link
🔨 Latest commit f529974

@netlify

netlify Bot commented Aug 4, 2026

Copy link
Copy Markdown

👷 Deploy request for nx-dev pending review.

Visit the deploys page to approve it

Name Link
🔨 Latest commit f529974

@sdjayna sdjayna changed the title fix(core): bound Windows watcher handle usage fix(core): disable output tracking watcher on Windows Aug 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Windows output tracking duplicates per-directory HANDLE cost across workspace

1 participant