|
| 1 | +--- |
| 2 | +title: Merge Atomized Playwright Outputs |
| 3 | +description: Learn how to enable output merging for atomized Playwright tests |
| 4 | +sidebar: |
| 5 | + label: Merge Atomized Outputs |
| 6 | +filter: 'type:Guides' |
| 7 | +--- |
| 8 | + |
| 9 | +When running Playwright tests in CI environments, Nx can [split tests across multiple parallel tasks](/docs/features/ci-features/split-e2e-tasks) for faster execution. |
| 10 | +However, this creates separate test reports for each task. |
| 11 | +With the Nx Plugin, the Playwright report merging feature allows you to combine these individual reports into a single, unified report. |
| 12 | + |
| 13 | +**Benefits:** |
| 14 | + |
| 15 | +- View all test results in one consolidated report |
| 16 | +- Maintain the same reporting experience as non-atomized tests |
| 17 | +- Keep the performance benefits of parallel test execution |
| 18 | + |
| 19 | +## Prerequisites |
| 20 | + |
| 21 | +- Nx workspace with `@nx/playwright` plugin |
| 22 | +- Playwright tests configured in your project |
| 23 | +- CI environment with [task atomization enabled](/docs/features/ci-features/split-e2e-tasks) |
| 24 | +- Nx v21.6.1 |
| 25 | + |
| 26 | +{% aside type="note" title="Nx Version"%} |
| 27 | +Nx v21.6.1 introduced automatic configuration of report merging which is the focus of this guide. |
| 28 | +Similar steps can be followed for manual configuration of Playwright for any versions prior to Nx v21.6.1 |
| 29 | +{%/aside%} |
| 30 | + |
| 31 | +## How report merging works |
| 32 | + |
| 33 | +1. **Blob Reports**: Playwright generates [lightweight "blob" reports](https://playwright.dev/docs/test-reporters#blob-reporter) during test execution |
| 34 | +2. **Individual Tasks**: Each atomized test task creates its own blob report |
| 35 | +3. **Report Merging**: A separate merge task combines all blob reports into unified output using your configured reporters (HTML, JSON, JUnit, etc.) |
| 36 | + |
| 37 | +## Configuration |
| 38 | + |
| 39 | +### Step 1: Enable blob reports |
| 40 | + |
| 41 | +The `nxE2EPreset` from `@nx/playwright/preset` automatically enable the blob reporter when running in CI: |
| 42 | + |
| 43 | +```ts |
| 44 | +// playwright.config.ts |
| 45 | +import { defineConfig } from '@playwright/test'; |
| 46 | +import { nxE2EPreset } from '@nx/playwright/preset'; |
| 47 | + |
| 48 | +export default defineConfig({ |
| 49 | + // automatically uses the blob reporter for CI |
| 50 | + ...nxE2EPreset(__filename, { testDir: './src' }), |
| 51 | + // Your additional configuration |
| 52 | +}); |
| 53 | +``` |
| 54 | + |
| 55 | +> The preset has a `generateBlobReports` option that can be used to disable the reporter or maybe enable it not just in CI. |
| 56 | + |
| 57 | +**Alternative manual configuration:** |
| 58 | + |
| 59 | +```ts |
| 60 | +// playwright.config.ts |
| 61 | +export default defineConfig({ |
| 62 | + reporter: [ |
| 63 | + ['html', { outputFolder: 'playwright-report' }], |
| 64 | + ...(process.env.CI ? [['blob', { outputDir: 'blob-report' }]] : []), |
| 65 | + ], |
| 66 | + // Your other configuration |
| 67 | +}); |
| 68 | +``` |
| 69 | + |
| 70 | +### Step 2: Configure the Playwright plugin |
| 71 | + |
| 72 | +Add a `ciTargetName` to the Playwright plugin to your `nx.json`. This will enable task splitting and report merging features for CI environments: |
| 73 | + |
| 74 | +```json |
| 75 | +// nx.json |
| 76 | +{ |
| 77 | + "plugins": [ |
| 78 | + { |
| 79 | + "plugin": "@nx/playwright/plugin", |
| 80 | + "options": { |
| 81 | + "targetName": "e2e", |
| 82 | + "ciTargetName": "e2e-ci" |
| 83 | + } |
| 84 | + } |
| 85 | + ] |
| 86 | +} |
| 87 | +``` |
| 88 | + |
| 89 | +With the above configurations, Nx automatically creates these targets for your Playwright projects: |
| 90 | + |
| 91 | +- `e2e`: Regular target for local development (runs all tests together) |
| 92 | +- `e2e-ci`: CI target that runs tests in parallel across multiple tasks |
| 93 | +- `e2e-ci--merge-reports`: Target that merges blob reports into unified reports |
| 94 | + |
| 95 | +## Running the feature |
| 96 | + |
| 97 | +**Local Development:** |
| 98 | + |
| 99 | +```shell {% frame="none" %} |
| 100 | +# Run tests normally (no blob reports generated) |
| 101 | +nx run my-app:e2e |
| 102 | +``` |
| 103 | + |
| 104 | +**CI Environment:** |
| 105 | + |
| 106 | +```shell {% frame="none" %} |
| 107 | +# Run atomized tests (generates blob reports) |
| 108 | +nx run my-app:e2e-ci |
| 109 | + |
| 110 | +# Merge the reports |
| 111 | +nx run my-app:e2e-ci--merge-reports |
| 112 | +``` |
| 113 | + |
| 114 | +## Complete example |
| 115 | + |
| 116 | +Here's a complete working example for reference: |
| 117 | + |
| 118 | +```ts |
| 119 | +// playwright.config.ts |
| 120 | +import { defineConfig } from '@playwright/test'; |
| 121 | +import { nxE2EPreset } from '@nx/playwright/preset'; |
| 122 | + |
| 123 | +export default defineConfig({ |
| 124 | + ...nxE2EPreset(__filename, { testDir: './e2e' }), |
| 125 | + use: { |
| 126 | + baseURL: 'http://localhost:4200', |
| 127 | + }, |
| 128 | + webServer: { |
| 129 | + command: 'npm run start', |
| 130 | + port: 4200, |
| 131 | + reuseExistingServer: !process.env.CI, |
| 132 | + }, |
| 133 | +}); |
| 134 | +``` |
| 135 | + |
| 136 | +```json |
| 137 | +// nx.json |
| 138 | +{ |
| 139 | + "plugins": [ |
| 140 | + { |
| 141 | + "plugin": "@nx/playwright/plugin", |
| 142 | + "options": { |
| 143 | + "targetName": "e2e", |
| 144 | + "ciTargetName": "e2e-ci" |
| 145 | + } |
| 146 | + } |
| 147 | + ] |
| 148 | +} |
| 149 | +``` |
| 150 | + |
| 151 | +## CI Integration |
| 152 | + |
| 153 | +> The following example CI configuration is for GitHub Actions, but the same steps can be followed in all CI providers. |
| 154 | +> The most important part is to always call the merge report target, `e2e-ci--merge-reports`, even if the E2E tests fail to always have a final merged report. |
| 155 | + |
| 156 | +```yaml |
| 157 | +// .github/workflows/ci.yaml |
| 158 | +name: CI |
| 159 | +on: |
| 160 | + pull_request: |
| 161 | + |
| 162 | +jobs: |
| 163 | + main: |
| 164 | + steps: |
| 165 | + ... |
| 166 | + - name: Run E2E Tests |
| 167 | + run: nx run my-app:e2e-ci |
| 168 | + |
| 169 | + - name: Merge Test Reports |
| 170 | + run: nx run my-app:e2e-ci--merge-reports |
| 171 | + # Run even if tests fail to get partial results |
| 172 | + if: always() |
| 173 | +``` |
| 174 | + |
| 175 | +## Troubleshooting |
| 176 | + |
| 177 | +### "The blob reporter is not configured" |
| 178 | + |
| 179 | +**Problem:** The merge task warns that no blob reporter is configured. |
| 180 | + |
| 181 | +**Solution:** Ensure the blob reporter is added to your Playwright configuration: |
| 182 | + |
| 183 | +- If overriding the `reporter` in the Playwright configuration, make sure to add the blob reporter: |
| 184 | + |
| 185 | +```ts |
| 186 | +// playwright.config.ts |
| 187 | +export default defineConfig({ |
| 188 | + reporter: [ |
| 189 | + // existing reporters |
| 190 | + ...(process.env.CI ? [['blob', { outputDir: 'blob-report' }]] : []), |
| 191 | + ], |
| 192 | + // Your other configuration |
| 193 | +}); |
| 194 | +``` |
| 195 | + |
| 196 | +### Missing test results in report |
| 197 | + |
| 198 | +**Problem:** The merge shows fewer results than expected test suites. |
| 199 | + |
| 200 | +**Solution:** |
| 201 | + |
| 202 | +- Check if some test tasks failed or were cancelled |
| 203 | +- Verify all blob report files are present in the blob report directory |
| 204 | +- Review CI logs for any task failures |
| 205 | + |
| 206 | +### Reports not merging |
| 207 | + |
| 208 | +**Problem:** Individual blob reports exist but aren't being merged. |
| 209 | + |
| 210 | +**Solution:** |
| 211 | + |
| 212 | +- Ensure the merge target is running after all test tasks complete |
| 213 | + - The default merge target name is `e2e-ci--merge-reports` |
| 214 | + - You can run `nx show project <e2e-project-name>` to view targets available on your project to verify the target exists |
| 215 | +- Check that the blob report directory path is correct |
| 216 | +- Verify Playwright CLI is available in the CI environment |
| 217 | + |
| 218 | +### [WARNING] "No additional reporters are configured" |
| 219 | + |
| 220 | +**Problem:** Only the blob reporter is configured, so it will be the only report produced |
| 221 | + |
| 222 | +**Solution:** Add at least one additional reporter (HTML, JSON, JUnit, etc.) alongside the blob reporter. |
| 223 | + |
| 224 | +List of available Playwright reports can be found in the [Playwright documentation](https://playwright.dev/docs/test-reporters). |
| 225 | + |
| 226 | +## Best Practices |
| 227 | + |
| 228 | +1. **Environment-based Configuration**: Only enable blob reports in CI to avoid overhead during local development |
| 229 | +2. **Error Handling**: Use `if: always()` (or similar) in CI workflows to merge partial results even when some tests fail |
| 230 | +3. **Output Artifacts**: Archive both blob reports and merged reports as CI artifacts for debugging |
| 231 | +4. **Validation**: The merge task automatically validates the expected number of test results |
![graphite-app[bot]](https://avatars.githubusercontent.com/in/158384?v=4&size=40){kind=avatar}
0 commit comments