[ZEPPELIN-6314] Set up a new E2E testing environment in the New UI

[dididy]

What is this PR for?

[Summary]

• Proposal: Migrate the existing Protractor (and Karma/Jasmine-based E2E/testing system) to a Playwright-based testing framework.
• Objective: Achieve a more stable and faster testing process while supporting modern E2E requirements such as cross-browser testing, multi-tab scenarios, and automatic waits. Additionally, leverage Playwright MCP / code generator to automate and accelerate test creation.

[Background]

1. Protractor has long been the standard for E2E testing in the Angular ecosystem. However, with the official deprecation/termination of support by the Angular team, long-term maintenance can no longer be relied upon.
2. Existing Protractor tests are WebDriver-based (control method, wait strategies, etc.), which often led to flaky test cases and increased maintenance costs.
3. The Karma + Jasmine-based unit testing environment, widely used as the default in Angular CLI, also has limitations in terms of build/compile costs, speed, and developer experience (configuration, parallel execution, etc.). Modern alternatives offer a better DX.
   • Since there were no test files, the related environment was removed from this branch, and any future unit tests will be migrated to Jest.

[Why Playwright?]

Playwright provides a robust ecosystem with stability, speed, standardization, and AI-assisted automation capabilities.

• Test Generator (Codegen)
  • Records user interactions in the browser and generates scaffolding for tests—useful for quickly setting up E2E scenarios.
• MCP (Model Context Protocol) Integration
  • Playwright has extensive documentation and examples for workflows that generate and validate tests via MCP (LLM ↔ test runner interface), enabling automated test creation and LLM-assisted refactoring. This reduces the need for fully manual test writing.
  • In the MCP era, Playwright is the most widely used tool.
    • According to State of JavaScript 2024, Playwright is the most widely used E2E tool at work.
• Functional Advantages
  • Multi-browser support: Chromium / Firefox / WebKit
  • Multi-tab & cross-domain (cross-origin) testing
  • Robust auto-waiting, parallel execution
  • Built-in reporters, traces, videos, snapshots
• Why Not Cypress?
  • Parallel execution requires paid Cypress Cloud
  • Cannot handle iframes, new tabs, multi-tab, or popup scenarioswithout tricks
  • MCP support is community-driven and not standardized
• Why Not Puppeteer?
  • Chrome-dependent, unsuitable for multi-environment E2E testing

[Project Structure Overview]

• fixtures/: Environment setup tools for test execution (e.g., tearup / teardown).
• models/: Contains POM (Page Object Model) classes for reusable page actions.
• tests/: Contains spec files for the actual test cases.
• helper.ts: Zeppelin-specific helper classes.
• reporter.coverage.*.ts: Custom reporters for checking test coverage.
• utils.ts: Test utilities and page constants.

Node 16 is no longer supported in @playwright/test versions after v1.53.2. As a result, this project uses v1.53.2 instead of the latest version (v1.55.0) to maintain compatibility.

[Coverage Rule]

Goal: Ensure that for each *.component.ts file, at least one test passes. In other words, verify that E2E tests cover the functionality of each component.

Approach (Proposed Flow)

1. Enable frontend code coverage collection by adding a custom reporter.
2. At the start of each test (or when entering a page), register coverage using a helper like testPageBeforeEach({TEST-PATH}). Example usage shown below.
3. During the test execution, if additional coverage is generated, register it via the testPage({TEST-PATH}, testInfo) helper.
4. In the Playwright custom reporter, after all tests have finished, parse the collected coverage data:
   • Read the list of files matching src/app/**/*.component.ts.
   • Verify that each component has test coverage
   • If any component is not sufficiently covered, fail the CI or generate a separate report.

Example: Using testPageBeforeEach and testPage

test.describe('TEST EXAMPLE', () => {
  testPageBeforeEach('src/app/app.component');
  ...
  test('TEST FOR LOGIN', async ({ page }, testInfo) => {
    testPage('src/app/pages/login/login.component', testInfo);
    ...
  });
  ...
});

[Test Generators / Leveraging MCP]

Automated Initial Scaffolding: Developers no longer need to manually write each scenario. Using Playwright Codegen / MCP, initial test templates can be generated automatically and then refined by humans, dramatically increasing productivity.

• Using Test Generator Options
  • You can specify viewport, device, color scheme, geolocation, locale, timezone, auth.json file, etc.
  • npm run e2e:codegen
• LLM-Assisted Test Generation
  • Through MCP can be provided with the page state to suggest or generate test cases, enabling rapid coverage expansion.
  • Developed by Microsoft, MCP integrates well with IDEs like VS Code, making AI-assisted features (test code auto-generation, scenario suggestions, etc.) seamless.
  • Example workflow connecting Playwright test runner with MCP server:
    • Read the webpage state → LLM generates test cases automatically
    • Page Object Model is auto-created/recommended
    • Scenario execution and result feedback can be integrated - Playwright MCP Integrations
  • Copilot / Claude / Cursor / Chrome Extension

[Reference]

• apache/apisix-dashboard
• Protractor deprecation: Angular Blog
• Playwright Codegen
• Playwright MCP / Generative test examples
• Playwright Migration Guide
• Playwright Reporter API / Custom reporters

What type of PR is it?

Improvement
Feature

Todos

• The account currently logs in as anonymous in the local environment, but I’m not sure how to perform a tear-up with a specific account. I’ll need to look into this further. If anyone is familiar with this, I would appreciate advice. <- I found conf/shifo.ini
• Regarding coverage measurement, we could consider adopting a better approach if one exists. From what I’ve found so far, there doesn’t seem to be a tool that can automatically detect and measure coverage.
• The GitHub Action is running too slowly, and it seems we could improve performance by using a caching strategy. <- Modify it so that it runs during Maven’s test phase and is referenced accordingly
• We should discuss the approach of moving the E2E tests to a separate repository.
• We need to consider how to contribute to E2E tests. Currently, the plan is as follows:
  • After this PR is merged, either use ZEPPELIN-6314 or create a. new parent issue, then manage contributions as sub-issues.
    • To avoid duplicate work, contributors can:
      1. Check the PAGES variable in utils to find areas they want to work on.
      2. Leave a comment and open a new sub-issue.
         • Example comment:
           • "I would like to work on testing the header area."
           • Path: src/app/share/header/header.component

What is the Jira issue?

• [ZEPPELIN-6314]

How should this be tested?

Screenshots (if appropriate)

Questions:

• Does the license files need to update? N
• Is there breaking changes for older versions? N
• Does this needs documentation? N

[tbonelee]

@Henry-Hong Could you help review this PR when you have the time?

[tbonelee]

If both commands must succeed to proceed, replace || with && or ;. With ||, the second command runs only if the first fails.

Also, if the first command succeeds, it blocks, so the second won't start. It's better to run them in parrallel and launch the second only after the first is up (e.g., using the start-server-and-test package).

[Henry-Hong]

Maybe you can try '&' operator.

But 'concurrently' would be also great option since '&' doesn't support cross-platform. (& is unix shell command)

Or we can just assume that user has already run angular dev server through 'npm run start' in another shell. So we can do just only "e2e:codegen" : "playwright codegen http://localhost:4200" without any operators and additional dependencies.

[dididy]

fcc9b57

It’s not a critical part, but installing a new dependency feels a bit awkward, and there’s also a nuance that codegen itself assumes a local server is already running. So I went with Henry-Hong’s suggestion and removed the part before the or operator.

[tbonelee]

Since this PR is not small, I'll be adding the comments along the way as I review the files. Thanks for your understanding.

[Henry-Hong]

Thank you for your PR!

I will take a look at it, it might take some time since it has quite big diffs 😲

[Henry-Hong]

What is the reason for slowMo: 200 only for webkit?

[dididy]

Safari(WebKit) can be less stable than other browsers in terms of event handling and rendering timing, which may cause flaky tests. To mitigate this, I applied slowMo: 200 to introduce a short delay between actions and improve test stability.

[Henry-Hong]

LGTM

[tbonelee]

Thank you for working on this job. LGTM👍

[tbonelee]

Could you rebase this onto the master branch?

[tbonelee]

Merged into master and branch-0.12