chore(repo): prebuild shared e2e base workspace to skip per-file create-nx-workspace - #35964
Draft
FrozenPandaz wants to merge 7 commits into
Draft
chore(repo): prebuild shared e2e base workspace to skip per-file create-nx-workspace#35964FrozenPandaz wants to merge 7 commits into
FrozenPandaz wants to merge 7 commits into
Conversation
👷 Deploy Preview for nx-docs processing.
|
✅ Deploy Preview for nx-dev ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
Contributor
|
View your CI Pipeline Execution ↗ for commit 40f3810
☁️ Nx Cloud last updated this comment at |
FrozenPandaz
force-pushed
the
e2e-shared-base-workspace
branch
from
June 13, 2026 03:18
fe5814a to
6fd5d03
Compare
FrozenPandaz
force-pushed
the
e2e-shared-base-workspace
branch
from
June 24, 2026 02:32
6fd5d03 to
cafa79b
Compare
…te-nx-workspace Each atomized e2e-ci task pays a ~40s create-nx-workspace cold start in its first newProject hook (the proj-backup template is per-process, so it never crosses files or agents). This adds a cached populate-e2e-base-workspace task that builds a bare workspace template once and outputs it, so Nx restores it to every agent. newProject seeds from the template (~0.3s copy) instead of running create-nx-workspace, falling back to the original build when the template is absent (e.g. pnpm).
…emplate newProject's seeding branch skipped runCreateWorkspace, which sets the module-level projName as a side effect that packageInstall -> getPackageManagerCommand relies on. Without it, the package.json read resolved to .../undefined/package.json and crashed, orphaning the seeded 'proj' dir and cascading into 'directory proj already exists' on subsequent newProject calls.
Declare the proj-backup template (a dependency output) via dependentTasksOutputFiles in e2eInputs so e2e tasks expect to read it, and use the /**/* glob for the populate task's script dir so its own script read is expected. Clears the sandbox violations introduced by the shared base workspace.
…orkspace master made oxfmt the default formatter for e2e workspaces so every suite exercises it. The prebuilt template still hardcoded prettier, so any suite seeded from the template would silently lose that coverage.
FrozenPandaz
force-pushed
the
e2e-shared-base-workspace
branch
from
September 1, 2026 22:00
cafa79b to
56258fd
Compare
…ually uses
The template was npm-only, but .nx/workflows/agents.yaml sets SELECTED_PM=pnpm
for the distributed agents that run every e2e-ci--* task. sharedBaseWorkspacePath
looked for a pnpm template, found none, and fell back to create-nx-workspace for
every spec file on Linux -- so the ~177MB artifact was restored per agent for no
benefit.
Build the template for SELECTED_PM instead of hardcoding npm, and declare
SELECTED_PM as an input so the cache can't hand an npm-built template to a pnpm
agent. Templates are built through a parallel map so adding a second package
manager stays a one-line change.
Also add the task to e2e-{maven,gradle,docker}, which declare their own dependsOn
and so opted out of the targetDefaults list entirely.
sharedBaseWorkspacePath returned null for any preset but 'apps', so the 17 spec files that start on the 'ts' preset always paid create-nx-workspace. Measured across the 150 spec files that call newProject(): 133 start on 'apps' (mostly by defaulting to it) and 17 on 'ts' -- nothing else. Building both covers all of them. Templates are now keyed <package-manager>/<preset> and the matrix is built in parallel, so adding a preset costs wall-clock only if it is genuinely a new combination. Directory existence is the only gate now, so an unlisted preset simply falls back. The long-tail presets (react-standalone, nuxt, angular-*) reach create-nx-workspace through runCreateWorkspace(), which the workspace-create suites use to exercise the real thing, and which this template must not replace.
…matrix newProject() call sites do not all use getSelectedPackageManager(): of 208, 185 do, while the rest pin a package manager or iterate all four (see affected-auto-lockfile.test.ts). Since backupPath is keyed on package manager, every pinned call paid its own create-nx-workspace -- migrate-registry.test.ts alone paid four. Build all four package managers against both presets so no call site falls back. SELECTED_PM is no longer an input: the output is identical whatever an agent selects, so keying on it would only split the cache into duplicate entries. Combinations a machine cannot build (bun or yarn missing) are logged and skipped; only an all-empty matrix fails the task, since a missing template is a fallback rather than an error.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Current Behavior
The e2e suite is atomized into one task per spec file (
e2e-ci--src/<file>.test.ts). Each task runsnewProject()in its firstbeforeAll/beforeEach, which performs acreate-nx-workspacecold start (~40s). On the tight hook timeout that intermittently flakes — it's the same root cause behind severalExceeded timeout of 60000 ms for a hookfailures.The existing
proj-backuptemplate that makes within-filenewProjectcalls fast is built lazily and lives under the per-process e2e root, so it never crosses files or distributed agents. Every spec file pays the cold start again.Expected Behavior
A new cached
populate-e2e-base-workspacetask builds a bare workspace template once, up front, and declares it as an Nx output (dist/local-registry/proj-backup). Because it's a cached output, Nx restores it to every distributed agent the same way the verdaccio storage already is — keyed on the nx build hash, so it rebuilds automatically when nx changes.newProject()then seeds the workspace from that template viacopySync(~0.3s) instead of runningcreate-nx-workspace(~40s). It falls back to the original build when the template is absent (e.g. pnpm, or the task didn't run), so the change is purely additive. Each test still installs its ownpackages[]on top of the bare base, so what ends up installed per workspace is unchanged.Measured locally
create-nx-workspace --preset=apps, npm).copySync~0.3s vscreate-nx-workspace~40s.Note for reviewers
The remaining trade-off is the ~177 MB artifact restored per agent per run; the net win depends on files-per-agent in the e2e-ci distribution, worth confirming with an A/B before merging. npm only for now (pnpm store symlinks aren't portable) — pnpm falls back to the original path.
Related Issue(s)
N/A — e2e infrastructure / performance.
View session information ↗