fix(ci): restore torch E2E coverage in unit-vs-E2E summary - #942
Open
WangLingxun wants to merge 4 commits into
Open
fix(ci): restore torch E2E coverage in unit-vs-E2E summary#942WangLingxun wants to merge 4 commits into
WangLingxun wants to merge 4 commits into
Conversation
PR #898 moved the torch coverage combine step into docker exec without passing COVERAGE_FILE, so the first combine merged .coverage.unit as a parallel shard instead of .coverage_e2e.*. Rename the unit snapshot to .coverage_unit, merge E2E shards explicitly, and warn when coverage_combined.json is missing.
WangLingxun
marked this pull request as ready for review
July 31, 2026 06:39
WangLingxun
requested review from
Xiaoming-AMD,
limou102 and
wenxie-amd
as code owners
July 31, 2026 06:39
- Add branch = true / --cov-branch to unit and E2E .coveragerc so a covered function means both branch arms ran, not just one line. - Only combine E2E shards when present, and require coverage_combined.json to be non-empty, so a failed merge shows red instead of silently succeeding with no file. - Caption the "including all modules" summary line as lines+branches, since coverage.py's percent_covered now folds branches in. - Refresh the stale backends/megatron note: CI E2E now runs 15+ configs/PR, not 1-2.
test_memory_benchmark SIGABRT'd across all 8 ranks in the last run: enabling coverage's branch trace while torchrun's real GPU training subprocesses run is not safe. Unit (a plain pytest process) keeps --cov-branch; only the E2E lanes drop back to line-only coverage.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
docker execwithout passingCOVERAGE_FILE..coverage.unitto.coverage_unitso it is not treated as a.coverage.*parallel shard..coverage_e2e.*shards before line-merging with unit data.coverage_combined.jsonto be non-empty, so a failed merge fails the step instead of silently succeeding with no file.--cov-branchfor the unit lane only, so a unit-tested function is only "covered" once both branch arms have run, not just one line of it. E2E stays line-only: turning onbranch = truein.coveragerc_e2emade the trainer subprocess chain crash (SIGABRTacross all 8 ranks intest_memory_benchmark) once coverage's branch trace was active during real GPU training — caught by this PR's own CI run, reverted before merge.percent_coverednow folds branches into the unit side of that number.backends/megatronnote incoverage_summary.py: CI E2E now runs 15+ configs/PR, not 1-2.