[quality] test: unit coverage for scripts/update_coverage_snapshot.py — new tests/unit/test_update_coverage_snapshot.py only - #776
Open
kubestellar-hive[bot] wants to merge 1 commit into
Conversation
scripts/update_coverage_snapshot.py is a required PR gate — pr-validate.yml runs it with --check to block stale suite-map coverage tables — but it had zero unit tests. A regression in its tag parsing or classification would silently change the scenario numbers every test PR is measured against. Adds tests/unit/test_update_coverage_snapshot.py: 44 tests covering parse_scenarios (feature-tag inheritance, per-scenario tag scoping, Scenario Outline counted once, comment lines, Rule:/Background:/Examples: tag resets), classify (quarantine > hardware_blocked/future/pending > active precedence), SuiteCounts.total, count_scenarios (suite name derived from the first path part under tests/, multi-file and multi-suite accumulation, non-.feature files ignored), render_snapshot (marker wrapping, totals line, sorted rows, column values, SUITE_NOTES lookup), update_file (missing-marker exit 2, write/idempotent/check modes, check never writes) and main (--repo-root and --check plumbing). Also asserts the committed snapshot is current, so the CI invariant is reproduced locally. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: sec-check[bot] <sec-check[bot]@users.noreply.github.com>
|
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Refs #775
Claimed ground (for disjointness checks)
tests/unit/test_update_coverage_snapshot.py(new)scripts/update_coverage_snapshot.py—parse_scenarios,classify,SuiteCounts.total,count_scenarios,render_snapshot,update_file,mainDisjoint from the open testsuite PRs: #774 (
scripts/compose-e2e-image.sh), #773 (SSH transport), #771 (Containerfile.runner SHA-256), #768 (offline_boot_steps.py/sudo_rs_steps), #766 (scripts/generate_skill_index.py), #765 (behave environment quarantine gate), #757 (input-remapper QEMU), #751 (screenshot_steps.py). None of them touchscripts/update_coverage_snapshot.pyor a file namedtests/unit/test_update_coverage_snapshot.py.Why
.github/workflows/pr-validate.yml:84runspython3 scripts/update_coverage_snapshot.py --checkas a required gate, andAGENTS.mdforbids hand-editing the numbers it generates — but the script itself had zero unit tests. The gate is self-referential: it diffs the script's output against a table the same script produced, so a tag-precedence or feature-tag-inheritance regression moves both sides together and CI stays green while the suite-map numbers drift.What this adds
44 tests in
tests/unit/test_update_coverage_snapshot.py:parse_scenariosScenario Outlinecounted once (not perExamples:row), comments ignored,Rule:/Background:/Examples:reset pending tags, hyphen/underscore tag names, file with no scenariosclassifypending/future/hardware_blocked→ pending,quarantineoutranks every backlog tagSuiteCountstotalsums all three buckets; zero defaultscount_scenariostests/, bucketing by precedence, multi-file accumulation, multi-suite separation, non-.featurefiles ignoredrender_snapshotSUITE_NOTESprose for known suites, empty note for unknown suites, header rowupdate_file--checkreturns 1 on stale and never writes,--checkreturns 0 when current,--checkcatches a newly added scenariomain--repo-roothonoured,--checkpropagates and does not write--checkassertion, reproduced inside the unit suiteAll fixtures build synthetic feature trees under
tmp_path; nothing touches the realdocs/skills/test-authoring/suite-map/SKILL.mdexcept the read-only invariant test.Verification
Full unit suite before and after this branch is identical —
3 failed, 9 errorsin both runs, all pre-existing and caused bybehave/webdriver not being installed in the local sandbox; passing count goes1236 → 1280(+44).Filed by quality agent (hold-gated mode). Human review required — do not merge without review.
— hive: agent=quality backend=copilot model=claude-opus-5