feat(test-runner): annotate serial suites for custom sharding

[Simon Knott (Skn0tt)]

Fixes #42143 with a solution that doesn't give prominent public API. At the same time it doesn't give the parallelMode === 'default' bit, which is also useful to sharding implementations. Gotta API-review this critically.

[github-actions]

Test results for "MCP"

2 failed
❌ [firefox] › mcp/cli-core.spec.ts:140 › uncheck @mcp-windows-latest-firefox
❌ [webkit] › mcp/http.spec.ts:144 › http transport browser sigint @mcp-windows-latest-webkit

8061 passed, 1284 skipped

---

Merge workflow run.

[github-actions]

Test results for "tests 1"

1 failed
❌ [playwright-test] › playwright.trace.spec.ts:346 › should retain traces for interrupted tests @windows-latest-node22

4 flaky

⚠️ [chromium-library] › library/video.spec.ts:736 › screencast › should work with video+trace `@chromium-ubuntu-22.04-node24`
⚠️ [chromium-library] › library/video.spec.ts:699 › screencast › should capture full viewport on hidpi `@chromium-ubuntu-22.04-node20`
⚠️ [firefox-library] › library/trace-viewer-scrub.spec.ts:177 › should drag scrubber to select action `@firefox-ubuntu-22.04-node20`
⚠️ [firefox-page] › page/page-emulate-media.spec.ts:144 › should keep reduced motion and color emulation after reload `@firefox-ubuntu-22.04-node20`

51240 passed, 1190 skipped

---

Merge workflow run.

[github-actions]

🔴 One failure is caused by this PR

Hi, I'm the Playwright bot and I took a look at the CI failures.

test-modifiers.spec.ts › static modifiers should be added in serial mode fails on all six OS/Node shards because this PR now adds a serial annotation to every test in a serial suite, and that test asserts exact annotation contents. The MCP failures are pre-existing flakes.

Details

Overall: one real, deterministic failure produced by this PR's own change; the MCP failures are unrelated low-rate flakes with history on other SHAs.

Caused by this PR

• [playwright-test] › test-modifiers.spec.ts › static modifiers should be added in serial mode (macos-node22, windows-node22, ubuntu-node20/22/24/26) — the change in suiteUtils.ts::bindFileSuiteToProject unshifts { type: 'serial' } onto test.annotations for any suite with _parallelMode === 'serial'. This test runs under test.describe.configure({ mode: 'serial' }) and asserts exact annotation arrays with .toEqual(...) (e.g. [{ type: 'slow', ... }], and [] for the last test), so every test now carries an extra serial entry and every assertion mismatches. Failing identically across all six shards with no engine divergence is the signature of a code-caused break, not a flake — and this test has zero failures anywhere else in the aggregated results DB. The PR updates this test's expectations in its own diff, so aligning them (add the serial entry) is the fix; the CI report predates the corrected build.

Pre-existing flake / infra

• [firefox] › mcp/cli-core.spec.ts:140 › uncheck — pre-existing flake. In the results DB this test failed 3 of 679 firefox runs, passed 676, on SHAs unrelated to this PR, and passes 100% on every other engine (chromium/chrome/msedge/webkit). This PR only touches the test-runner serial annotation and doesn't exercise the MCP uncheck path.
• [webkit] › mcp/http.spec.ts:144 › http transport browser sigint — pre-existing flake. Failed 1 of 691 webkit runs, passed 690, on unrelated SHAs; passes 100% on every other engine. Nothing in this PR touches the MCP HTTP transport.

The earlier MCP report's failures (mcp/http.spec.ts:105 › http transport browser lifecycle (isolated), mcp/cli-core.spec.ts:43 › click button) are the same story — both flip verdict at low rates across unrelated runs in the DB (lifecycle: failed 5/686 chromium, 3/691 webkit, etc.; click button: 1/679 firefox) and diverge between MCP runs, which is the flake signature. The 5 flaky entries in the "tests 1" report (chromium.spec.ts:177, video.spec.ts:736/356/495, ui-mode-trace.spec.ts:827) all passed on retry in code this PR doesn't touch — not part of the verdict.

_{Triaged by the Playwright bot - agent run}

[github-actions]

🟢 CI is clear — the remaining failures are pre-existing flakes

Hi, I'm the Playwright bot and I took a look at the CI failures.

The one real, PR-caused break (test-modifiers.spec.ts › static modifiers should be added in serial mode) is already fixed inside this PR's own diff — the latest "tests 1" report shows it passing. Everything still red in the most recent reports is a pre-existing flake, confirmed failing on runs this PR can't be responsible for.

Details

Overall: no failure in the latest reports is caused by this PR. The serial-annotation change did break test-modifiers (it asserts exact annotation arrays), but the PR updates those expectations in its own diff and the test now passes. The three failures in the newest "tests 1" and "MCP" reports each flake at low rates across unrelated SHAs.

Pre-existing flake / infra

• [playwright-test] › playwright.trace.spec.ts:346 › should retain traces for interrupted tests (windows-latest-node22) — flake. In the results DB it failed 1 of 958 runs, passed 957, and the failures land on the same windows-latest-node22 bot on unrelated work: a mainline push (sha 40372734, no PR) and PR fix(chromium): keep closing the target so page.close() cannot hang on a navigation commit #41912, i.e. places this PR can't touch. This PR only adds a serial annotation in suiteUtils.ts; it doesn't affect trace retention for interrupted tests.
• [firefox] › mcp/cli-core.spec.ts:140 › uncheck — flake. Failed 3 of 679 firefox runs, passed 676, on SHAs unrelated to this PR, and passes on every other engine. The PR doesn't exercise the MCP uncheck path.
• [webkit] › mcp/http.spec.ts:144 › http transport browser sigint — flake. Failed 1 of 691 webkit runs, passed 690, on unrelated runs; passes elsewhere. Nothing in this PR touches the MCP HTTP transport.

Earlier MCP failures (mcp/http.spec.ts:105 › http transport browser lifecycle (isolated), mcp/cli-core.spec.ts:43 › click button) are the same story — the lifecycle test flips verdict across engines (webkit 4/691, chromium 4/682, firefox 2/679, msedge 1/443) on unrelated SHAs. The flaky entries in the "tests 1" reports (chromium.spec.ts:177, video.spec.ts, trace-viewer-scrub.spec.ts:177, page-emulate-media.spec.ts:144, ui-mode-trace.spec.ts:827) all passed on retry in code this PR doesn't touch.

_{Triaged by the Playwright bot - agent run}