[Bug]: nativeWatcher does not detect file changes for lazy-compiled modules

[T9-Forever]

System Info

• @rspack/core: 1.6.x
• @rsbuild/core: 1.6.6
• OS: macOS 14.x
• Node.js: 20.x
• Project size: ~4700 file dependencies, ~10600 missing dependencies

Details

Steps to Reproduce

1. Enable both lazyCompilation and nativeWatcher
2. Start dev server: npm run dev
3. Open browser, navigate to a lazy-loaded route (triggers lazy compilation)
4. Modify a file in that lazy-loaded module
5. Expected: HMR update, page reflects changes
6. Actual: Nothing happens, no rebuild triggered

Workaround

Disable either lazyCompilation OR nativeWatcher:

• lazyCompilation: false + nativeWatcher: true → Works ✅
• lazyCompilation: true + nativeWatcher: false → Works ✅

Debugging Process

Step 1: Add Debug Logs to Native Watcher (Rust)

Modified rspack/crates/rspack_watcher/src/trigger.rs:

pub fn on_event(&self, path: &ArcPath, kind: FsEventKind) {
  let finder = self.finder();
  let associated_event = finder.find_associated_event(path, kind);

  if associated_event.is_empty() {
    eprintln!("[NativeWatcher] ⚠️ FILE CHANGE NOT DETECTED: {:?} {}", kind, path.display());
    eprintln!("[NativeWatcher]    Registered: files={}, dirs={}, missing={}",
      finder.files.len(), finder.directories.len(), finder.missing.len());
  } else {
    eprintln!("[NativeWatcher] ✅ File change detected: {:?} {} -> {} events",
      kind, path.display(), associated_event.len());
  }

  self.trigger_events(associated_event);
}

Step 2: Add Debug Logs to FileCounter (Rust)

Modified rspack/crates/rspack_core/src/utils/file_counter/mod.rs:

pub fn add_files(&mut self, resource_id: &ResourceId, paths: &ArcPathSet) {
  for path in paths {
    let list = self.inner.entry(path.clone()).or_default();
    let path_str = path.to_string_lossy();

    // Track specific file for debugging
    if path_str.contains("PersonaSetupEntry") {
      if list.is_empty() {
        eprintln!("[FileCounter] 🎯 NEW PersonaSetupEntry file added: {}", path_str);
      } else {
        eprintln!("[FileCounter] ⚠️ PersonaSetupEntry ALREADY EXISTS (not marked as added): {}", path_str);
        eprintln!("[FileCounter]    Existing resource_ids: {:?}", list);
      }
    }

    if list.is_empty() {
      self.incremental_info.mark_as_add(path);
    }
    list.insert(resource_id.clone());
  }
}

Step 3: Add Debug Logs to JavaScript Side

Modified rspack/packages/rspack/src/NativeWatchFileSystem.ts:

formatWatchDependencies(changes: WatchOutputFileSystemChanges) {
  const added = changes.addedFileDependencies?.length || 0;
  const removed = changes.removedFileDependencies?.length || 0;
  
  if (added > 0 || removed > 0) {
    console.log(`[NativeWatcher] 📝 Incremental update: +${added} -${removed} files`);
  }
  // ...
}

Step 4: Compile Debug Build

cd /path/to/rspack
pnpm run build:binding:dev
pnpm run build:js

Debug Findings

Initial Compilation (without lazy module access)

[FileCounter] 🎯 NEW PersonaSetupEntry file added: /Users/.../PersonaSetupEntry
[FileCounter] 🎯 NEW PersonaSetupEntry file added: /Users/.../PersonaSetupEntry/index.tsx
[FileCounter] 🎯 NEW PersonaSetupEntry file added: /Users/.../PersonaSetupEntry/index.ts
[FileCounter] 🎯 NEW PersonaSetupEntry file added: /Users/.../PersonaSetupEntry.tsx
... (various extensions tried as missing dependencies)

The file PersonaSetupEntry/index.tsx is registered as a NEW file dependency.

After Lazy Compilation Triggers (navigate to route)

start   building PersonaOnboard/PersonaSetupEntry/index.tsx
[FileCounter] ⚠️ PersonaSetupEntry ALREADY EXISTS (not marked as added): /Users/.../PersonaSetupEntry/index.tsx
[FileCounter]    Existing resource_ids: {Dependency(DependencyId(100972))}
[NativeWatcher] 📝 Incremental update: +43 -0 files
[NativeWatcher] 📝 Incremental update: +56 -0 files

Key observation: The file is NOT marked as "added" because it already exists in FileCounter!

After File Modification

[NativeWatcher] ⚠️ FILE CHANGE NOT DETECTED: Change /Users/.../PersonaSetupEntry/index.tsx
[NativeWatcher]    Registered: files=4728, dirs=0, missing=10658
[NativeWatcher] ⚠️ FILE CHANGE NOT DETECTED: Change /Users/.../PersonaSetupEntry/index.tsx
[NativeWatcher]    Registered: files=4728, dirs=0, missing=10658

The native watcher receives the file system event, but it cannot find the file in its registered set!

Root Cause Analysis

The Problem

1. Initial compilation: File dependencies (including lazy modules' entry points) are registered in FileCounter as missing_dependencies (because the module hasn't been compiled yet, only referenced)
2. Lazy compilation triggers: When user navigates to the route, lazy compilation kicks in and tries to add the file to FileCounter again
3. FileCounter logic

   if list.is_empty() {
     self.incremental_info.mark_as_add(path);  // Only mark as "added" if NEW
   }
   list.insert(resource_id.clone());

Since the file already exists (from initial missing_dependencies), it's NOT marked as "added"
4. Incremental update: The native watcher only receives files marked as "added" in incremental_info
5. Result: The file path is not in the native watcher's active watch set, so changes are not detected

Flow Diagram

Initial Compilation:
  PersonaSetupEntry/index.tsx → added to FileCounter as missing_dependency
  → marked as "added" in incremental_info
  → sent to native watcher ✅

Lazy Compilation:
  PersonaSetupEntry/index.tsx → already exists in FileCounter
  → NOT marked as "added" (list not empty)
  → NOT sent to native watcher ❌

File Change:
  Native watcher receives FS event
  → checks if path is in registered files
  → NOT FOUND (was never properly registered after lazy compilation)
  → change ignored ❌

Why It Works Without lazyCompilation

Without lazy compilation, all modules are compiled upfront:

1. All file dependencies are properly registered as file_dependencies (not just missing)
2. The incremental update correctly includes all files
3. Native watcher has all paths registered

Why It Works Without nativeWatcher

The default JavaScript-based watcher (watchpack) uses a different mechanism:

1. It watches directories, not individual files
2. It doesn't rely on the incremental added_files list
3. File changes are detected through directory watching

Proposed Fix Direction

Option 1: Fix FileCounter Incremental Logic

When lazy compilation adds a file that was previously a missing_dependency, it should be treated as a new file_dependency and marked as "added":

pub fn add_files(&mut self, resource_id: &ResourceId, paths: &ArcPathSet) {
  for path in paths {
    let list = self.inner.entry(path.clone()).or_default();
    
    // Check if this is transitioning from missing to file dependency
    let is_new_file_dep = list.is_empty() || self.is_transitioning_from_missing(path);
    
    if is_new_file_dep {
      self.incremental_info.mark_as_add(path);
    }
    list.insert(resource_id.clone());
  }
}

Option 2: Handle lazyCompilation Invalidation Specially

When compiler.watching.invalidate() is called by lazy compilation, ensure the changed files are explicitly passed to the native watcher.

Reproduce link

No response

Reproduce Steps

npm run dev