Skip to content

cleanup(core): move the affected touched-project locators into rust - #36825

Draft
AgentEnder wants to merge 4 commits into
feat/nxc-4861-files-inputfrom
feat/nxc-4859-task-based-affected
Draft

cleanup(core): move the affected touched-project locators into rust#36825
AgentEnder wants to merge 4 commits into
feat/nxc-4861-files-inputfrom
feat/nxc-4859-task-based-affected

Conversation

@AgentEnder

Copy link
Copy Markdown
Member

Stacked on feat/nxc-4861-files-input (NXC-4858). Review only the last commit; the base branch's 74 commits are not part of this change.

Groundwork for NXC-4859 (task-level nx affected). This is "PR 0": move the locators, change nothing else.

Current Behavior

filterAffected runs four TouchedProjectLocators in TypeScript, then reverse-walks the project graph. Three of them (getTouchedProjects, getImplicitlyTouchedProjects, getTouchedProjectsFromProjectGlobChanges) are pure path and glob matching, and they re-run minimatch per (pattern, file) pair — minimatch recompiles the pattern on every call, so the cost grows with changeset size times the number of distinct {workspaceRoot} filesets.

Expected Behavior

Those three move to a native locateTouchedProjects. Same results, same locator order, same duplicate-emitting contract.

The lockfile, npm-package and tsconfig locators stay in TypeScript — they depend on the npm/yarn/pnpm/bun lockfile parsers. They cross the boundary as ThreadsafeFunction callbacks invoked once per run, so the locator set is still extensible from JS rather than hardcoded.

Graph pruning also stays in TypeScript: the native ProjectGraph models dependencies as HashMap<String, Vec<String>>, with no edge type or source, so it cannot rebuild what filterAffected returns.

Also deletes workspace-json-changes.ts — the angular.json locator was already unregistered and has no remaining referents anywhere in the repo.

Behavior deltas worth knowing

Small, and all of them widen or correct rather than break:

  • {workspaceRoot}/config/ now matches config/app.json; minimatch did not.
  • {workspaceRoot}/!(dist)/** no longer matches dist/a.ts; minimatch's leading-! handling did.
  • A {workspaceRoot}/some-dir/ fileset becomes some-dir/**.
  • The old implicits[input] ??= [] on a plain object silently dropped any fileset named {workspaceRoot}/__proto__, constructor or toString, because the inherited property is non-nullish. The BTreeMap removes that quirk.
  • A self-referencing named input used to overflow the stack; it now terminates.

Performance

Measured on a synthetic 3000-project graph (30k files, 10k external nodes), min of 11, comparing this branch against its base. The trade is a variable cost for a fixed one, so which way it goes depends on how many distinct {workspaceRoot} filesets a workspace declares.

Many distinct filesets — the nx repo itself has 151 across 149 projects:

changed files before after
10 373 ms 65 ms
100 1426 ms 172 ms
1000 12824 ms 1239 ms

Few, shared filesets:

changed files before after
1 8.7 ms 40.4 ms
1000 59.3 ms 47.0 ms

So a large win where the pattern set is big, and a ~30 ms regression on a small-pattern workspace, from the fixed cost of walking the graph natively. filterAffected also marshals the project graph, which is memoized per graph identity — nx release calls filterAffected once per commit, and without the memo that re-marshalled the whole graph every time.

Related Issue(s)

NXC-4859

@nx-cloud

nx-cloud Bot commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

Nx Cloud AI Fix could not be generated

View fix generation details ↗


View your CI Pipeline Execution ↗ for commit 55d1656

Command Status Duration Result
nx affected --targets=lint,oxlint,test,build,e2... ❌ Failed 11m 19s View ↗
nx run-many -t check-imports check-lock-files c... ✅ Succeeded 4s View ↗
nx-cloud record -- pnpm nx-cloud conformance:check ✅ Succeeded 54s View ↗
nx build workspace-plugin ✅ Succeeded <1s View ↗
nx-cloud record -- nx sync:check ✅ Succeeded 18s View ↗
nx-cloud record -- nx format:check ✅ Succeeded <1s View ↗

☁️ Nx Cloud last updated this comment at 2026-08-30 02:39:57 UTC

Ports getTouchedProjects, getImplicitlyTouchedProjects and
getTouchedProjectsFromProjectGlobChanges to a native locateTouchedProjects.
No behavior change: the existing affected specs pass untouched.

The lockfile, npm-package and tsconfig locators stay in TypeScript because
they depend on the npm/yarn/pnpm/bun lockfile parsers. They are passed across
the boundary as ThreadsafeFunction callbacks and invoked once per run, so the
locator set stays extensible from JS.

Graph pruning stays in TypeScript too: the native ProjectGraph models
dependencies as HashMap<String, Vec<String>>, with no edge type or source, so
it cannot rebuild what filterAffected returns.

Also deletes workspace-json-changes.ts, the angular.json locator that was
already unregistered and had no remaining referents.
@AgentEnder
AgentEnder force-pushed the feat/nxc-4859-task-based-affected branch from 9f62051 to d191117 Compare August 28, 2026 20:01
getTaskOutputs and getExpandedTaskInputs both call loadIoSnapshotsForHead,
which reads a bundle off disk. Neither spec mocked it, so the snapshot branch
was skipped locally, where no bundle exists, and taken on CI, where Nx Cloud
has fetched one. Taking it reached customHasherTaskIds, whose call to
readProjectsConfigurationFromProjectGraph the project-graph mock does not
provide, and 5 tests across the two files failed.

Pin the bundle absent so the result no longer depends on ambient cache state.
Confirmed by forcing the bundle present locally, which reproduces the CI
failure exactly.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant