Skip to content

fix(core): hash task outputs under scoped directories - #36846

Open
jcaracciolo wants to merge 6 commits into
nrwl:masterfrom
jcaracciolo:fix/scoped-output-glob-hashing-minimal
Open

fix(core): hash task outputs under scoped directories#36846
jcaracciolo wants to merge 6 commits into
nrwl:masterfrom
jcaracciolo:fix/scoped-output-glob-hashing-minimal

Conversation

@jcaracciolo

@jcaracciolo jcaracciolo commented Aug 28, 2026

Copy link
Copy Markdown

Current Behavior

The native dependent-task output hasher derives an incorrect filesystem root for output globs containing scoped directory segments such as packages/@acme/producer/dist/**/*.d.ts. The glob parser's invalid-group recovery discards the ungrouped @, causing get_files_for_outputs to walk packages/acme/... and omit the generated files from the dependent task hash.

Expected Behavior

Output-root partitioning preserves literal ungrouped special characters while deriving static path segments, so scoped declarations are discovered and included in dependent task hashes. Normal glob conversion retains its existing invalid-group compatibility behavior, and valid grouped patterns such as @(foo|bar) continue to be converted normally.

The change adds regression coverage for scoped output expansion and verifies that changing a declaration beneath a scoped directory changes the task output hash.

Design Decisions

Why the parser has two behaviors

Nx intentionally discards ungrouped ?, +, and @ characters during normal glob conversion. This behavior was introduced in #21027 to tolerate malformed extglob prefixes such as +spec.ts instead of failing the entire glob. Existing workspaces may depend on that compatibility behavior.

Output-root partitioning has a different requirement: it must retain the original path text long enough to identify the directory that should be walked. In a path such as packages/@acme/producer/dist/**/*.d.ts, @acme is a literal directory segment rather than malformed extglob syntax.

The parser therefore accepts an explicit behavior:

  • Discard preserves the existing parse_glob behavior used by glob conversion.
  • Preserve retains ungrouped special characters for static-root partitioning.

Both behaviors use the same parser implementation; the mode only controls whether an ungrouped special character is retained in the parse result.

Alternatives considered

  • Preprocessing scoped paths with temporary marker strings was rejected because it duplicated part of the parser's syntax handling and required a separate restoration step.
  • Changing parse_glob to always preserve these characters was rejected because it would alter the established invalid-group contract and require changing existing compatibility tests.
  • Escaping every @ was rejected because valid forms such as @(foo|bar) must remain extglob expressions.

Compatibility and scope

  • Existing invalid-group tests and behavior remain unchanged.
  • Existing callers of parse_glob continue to use Discard behavior by default.
  • Only output-root partitioning requests Preserve behavior.
  • Regression tests are additive and cover scoped output discovery and task-output hash invalidation.

Related Issue(s)

Fixes #36845

@jcaracciolo
jcaracciolo requested a review from a team as a code owner August 28, 2026 23:33
@jcaracciolo
jcaracciolo requested a review from JamesHenry August 28, 2026 23:33
@netlify

netlify Bot commented Aug 28, 2026

Copy link
Copy Markdown

👷 Deploy request for nx-docs pending review.

Visit the deploys page to approve it

Name Link
🔨 Latest commit 16badf8

@netlify

netlify Bot commented Aug 28, 2026

Copy link
Copy Markdown

👷 Deploy request for nx-dev pending review.

Visit the deploys page to approve it

Name Link
🔨 Latest commit 16badf8

@AgentEnder

Copy link
Copy Markdown
Member

Thanks for sending this in. Assigned, CI is starting now, and someone will review as time allows.

Triage notes (automated triage, reviewed before posting)

The author explains why parse_glob keeps its discard behaviour and only output-root partitioning opts into preserving, which forecloses the broader change a reviewer would reach for first. The diff sits in native/cache, native/glob and the task hashers.

The SUCCESS rollup is the socket-security check alone rather than a CI verdict. CI, CodeQL, PR Title Validation and Unmergeable Labels Check are all sitting at action_required on the head commit, waiting on a maintainer to allow the runs, which triage is doing.

A closing reference to #36845, so merging this closes it.

@nx-cloud

nx-cloud Bot commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

View your CI Pipeline Execution ↗ for commit 16badf8

Command Status Duration Result
nx affected --targets=lint,oxlint,test,build,e2... ✅ Succeeded 55m 15s View ↗
nx run-many -t check-imports check-lock-files c... ✅ Succeeded 3s View ↗
nx-cloud record -- pnpm nx-cloud conformance:check ✅ Succeeded 37s View ↗
nx build workspace-plugin ✅ Succeeded <1s View ↗
nx-cloud record -- nx sync:check ✅ Succeeded 14s View ↗
nx-cloud record -- nx format:check ✅ Succeeded <1s View ↗

☁️ Nx Cloud last updated this comment at 2026-09-10 02:52:18 UTC

@nx-cloud nx-cloud Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nx Cloud has identified a flaky task in your failed CI:

Since the failure was identified as flaky, the solution is to rerun CI. Because this branch comes from a fork, it is not possible for us to push directly, but you can rerun by pushing an empty commit:

git commit --allow-empty -m "chore: trigger rerun"
git push

Nx Cloud View detailed reasoning in Nx Cloud ↗

🔔 Heads up, your workspace has pending recommendations ↗ to auto-apply fixes for similar failures.


🎓 Learn more about Self-Healing CI on nx.dev

@jcaracciolo

Copy link
Copy Markdown
Author

Seems i hit a flaky test @AgentEnder , what is the process to either re-run it or validate my changes against it?

Juan Franco Caracciolo added 4 commits September 4, 2026 13:59
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 923b49ad-c037-401e-aaff-a79945209bcd
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 923b49ad-c037-401e-aaff-a79945209bcd
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 923b49ad-c037-401e-aaff-a79945209bcd
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 923b49ad-c037-401e-aaff-a79945209bcd
@AgentEnder
AgentEnder force-pushed the fix/scoped-output-glob-hashing-minimal branch from a691a0a to f43ce63 Compare September 4, 2026 17:59
Juan Franco Caracciolo and others added 2 commits September 8, 2026 18:29
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@jcaracciolo

Copy link
Copy Markdown
Author

@FrozenPandaz what is the process for getting this reviewed? Would like to unblock this if possible

@AgentEnder

Copy link
Copy Markdown
Member

@FrozenPandaz what is the process for getting this reviewed? Would like to unblock this if possible

We'll get back to this, just have a lot of open PRs right now and are all decently busy on getting a few high priority things over the line. We are working on it, and appreciate the PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

scope: core core nx functionality type: bug

Projects

None yet

Development

Successfully merging this pull request may close these issues.

dependentTasksOutputFiles ignores outputs under scoped directories

3 participants