-
Notifications
You must be signed in to change notification settings - Fork 228
feat(ci): enable parallel test execution with sharding #2275
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
@kunal-511 PTAL! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
This PR implements parallel test execution with sharding to dramatically reduce Playwright test execution time from ~60 minutes to ~15-20 minutes. It achieves this through 4-way test sharding across 3 browsers (12 parallel jobs total) with improved wait strategies and timeout configurations for better CI reliability.
Key Changes:
- Implements 4-way test sharding with blob reporter for result merging
- Switches from 'networkidle' to 'domcontentloaded' wait strategy for faster, more reliable page loads
- Adds dynamic worker configuration (4 workers per shard in CI, 100% CPU utilization locally)
- Fixes Dependabot npm directory from root to
/frontend
Reviewed changes
Copilot reviewed 9 out of 9 changed files in this pull request and generated 10 comments.
Show a summary per file
| File | Description |
|---|---|
frontend/playwright.config.ts |
Adds worker count logic, timeout configurations (60s test, 10s expect, 15s action, 30s navigation), and blob reporter for sharding support |
frontend/playwright.global-setup.ts |
Implements waitForPageReady with fallback strategies and switches warmup from networkidle to domcontentloaded |
frontend/e2e/pages/base/BasePage.ts |
Changes default wait state from networkidle to domcontentloaded |
frontend/e2e/pages/ITSPage.ts |
Updates waitForReady to use domcontentloaded with retry logic for table visibility |
frontend/e2e/Install.spec.ts |
Replaces networkidle with domcontentloaded and adds content check via waitForFunction |
frontend/e2e/ObjectExplorerNavigation.spec.ts |
Switches to domcontentloaded with additional timeout buffers |
.github/workflows/playwright.yml |
Implements 4-way sharding matrix, merge-reports job, and conditional artifact uploads |
.github/labeler.yml |
Adds tests and helm label patterns |
.github/dependabot.yml |
Fixes npm directory, adds GitHub Actions ecosystem, and groups minor/patch updates |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
@clubanderson Please merge this after all ci passes |
|
ok - not passing yet |
|
hmm.. some test are failing because of Replace networkidle with domcontentloaded.. ill fix it tommrow |
|
have to rerun the whole loaclly then fix |
|
but parallel test execution with sharding worked 12 job |
|
from 60 min to 20 min |
|
@Arpit529Srivastava @alokdangre can you guys help in this.. with this PR test files and fix.. commit in this same pr |
|
login is failing majorly.. |
|
@alokdangre @Arpit529Srivastava there are more then 15 flasky test are there in shard 2/4.. you guys need to fix it or remove it |
|
you guys can test like |
|
@btwshivam i have added commits to this pr and made a pr in your fork plzz check it and run the ci also to check the if it works |
|
@alokdangre i cant see your comments.. u need to cclick on finish review button |
Fixes for CI
|
@alokdangre @Arpit529Srivastava still failing.. |
|
@alokdangre The global setup warmup is failing because networkidle timeout is being reached. The main issue is that the app uses MSW for mocking and may have pending connections. hve to use domcontentloaded instead of networkidle |
I have added few more commits as you said |
|
the tests are failing becuase of timeout, the content of pages are taking time to load, this is so random, like we can see, the tests which passed earlier failed now (with no changes made to them) and which failed earlier passed now, |
|
Alright!.. Guys Finally Worked.. |
|
@clubanderson It Worked 60 min to 15 min Transformation.. @alokdangre Thanks for the help! |
|
@clubanderson Merge it! |
|
/ok-to-test |
|
/lgtm |
|
@btwshivam: you cannot LGTM your own PR. DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: btwshivam The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
Playwright Configuration
Global Setup Improvements
GitHub Workflow (playwright.yml)
Test File Fixes
Dependabot & Labeler
Performance: ~60min -> ~15-20min (12 parallel jobs)"