ui: Start creating e2e tests for the UI#3930
Conversation
b5c47b1 to
3e217d9
Compare
|
@sschuberth @lamppu This has a problem that I don't know enough of GitHub environment to say, in which address the UI actually is when running the tests. It doesn't seem to be in localhost:8082, or localhost:5173, I tested both. The CI part is a bit of a mystery to me. Ah, it's probably just that the back-end isn't up at all when running the UI tests, that's why it doesn't answer. Need to think about this more. It seems quite inefficient if I need to build and run the core image separately just for some light UI tests. |
fe46128 to
a4092bf
Compare
|
@mnonnenmacher can you please help me with running this test on GitHub? The test needs core module being up and running, but I cannot get the test to run at all on GitHub. Locally, it runs and passes. I must admit, I really do not know anything about GitHub as the environment to run e2e tests on, but basically, the test is VERY simple: when core is running, go to UI, log in as an admin, and verify that the UI front page appears (by doing some very trivial tests that certain UI elements exist). |
a4092bf to
b4ce17f
Compare
There was a problem hiding this comment.
Pull Request Overview
This PR introduces an end-to-end (e2e) testing framework using Playwright for the UI component. The goal is to catch UI-breaking changes early with basic smoke tests.
Key Changes:
- Adds Playwright as a test framework with a basic smoke test that validates the login flow
- Configures the test environment to distinguish between unit tests and e2e tests
- Implements a CI workflow job that runs e2e tests against a live backend
Reviewed Changes
Copilot reviewed 7 out of 8 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| ui/vite.config.ts | Configures Vitest to exclude e2e tests from unit test runs |
| ui/tsconfig.node.json | Adds TypeScript configuration for Playwright config file |
| ui/tests/smoke-test.spec.ts | Implements a basic login flow smoke test |
| ui/playwright.config.ts | Configures Playwright test runner with browser settings |
| ui/package.json | Adds Playwright dependency and test:e2e script |
| ui/.gitignore | Excludes Playwright-generated files from version control |
| .github/workflows/build-and-test.yml | Adds CI job to run e2e tests with Docker backend |
Files not reviewed (1)
- ui/pnpm-lock.yaml: Language not supported
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
|
||
| - name: Run Playwright tests | ||
| env: | ||
| PLAYWRIGHT_TEST_BASE_URL: http://127.0.0.1:8082 |
There was a problem hiding this comment.
The environment variable 'PLAYWRIGHT_TEST_BASE_URL' is defined but the playwright.config.ts uses 'process.env.CI' to determine baseURL. This environment variable appears unused. Either use it in the config or use the CI-based logic consistently.
| PLAYWRIGHT_TEST_BASE_URL: http://127.0.0.1:8082 |
b4ce17f to
09d6e20
Compare
Do I read the test logs correctly that it fails to load the Keycloak login page? |
5368a50 to
125840c
Compare
At some point this was the issue, but I've now progressed to the point where I can log in as admin/admin. The problem remaining is that I cannot seem to get the UI yet to respond. Debugging. |
23f1400 to
9189b5e
Compare
c659000 to
c69d83a
Compare
Install Playwright [1] to handle e2e testing with the ORT Server UI. [1]: https://playwright.dev/docs/intro#installing-playwright Signed-off-by: Jyrki Keisala <jyrki.keisala@doubleopen.org>
c69d83a to
e0c1f39
Compare
The job: - starts core (plus keycloak, postgres, rabbitmq) services - waits for the core to respond by polling the `/liveness` endpoint response - waits for the UI to respond by looking for a specific message from Vite, then verifying that the UI actually returns HTML content - runs the Playwright e2e tests for the UI Playwright tests can take long to prepare and run (especially due to the need to install all Playwright browsers), so make the job completely independent of the `build-ui`, to be able to start the job early. Signed-off-by: Jyrki Keisala <jyrki.keisala@doubleopen.org>
Login to the ORT Server UI as an admin, and assert that some elements from the front page are rendered. This works as a crude smoke test for the UI; if something totally breaks the UI, the front page won't even load, and the test will fail. Resolves #3789. Signed-off-by: Jyrki Keisala <jyrki.keisala@doubleopen.org>
e0c1f39 to
8da0e8f
Compare
The purpose is not to start creating elaborate e2e tests, but as has been seen many times, it is possible to create a PR which totally breaks the UI, so best to implement the e2e test framework and run at least some trivial UI tests.
Please see the commits for details.