Skip to content

[quality] test: unit coverage for scripts/update_coverage_snapshot.py — new tests/unit/test_update_coverage_snapshot.py only - #776

Open
kubestellar-hive[bot] wants to merge 1 commit into
mainfrom
quality/test-update-coverage-snapshot
Open

[quality] test: unit coverage for scripts/update_coverage_snapshot.py — new tests/unit/test_update_coverage_snapshot.py only#776
kubestellar-hive[bot] wants to merge 1 commit into
mainfrom
quality/test-update-coverage-snapshot

Conversation

@kubestellar-hive

Copy link
Copy Markdown
Contributor

Refs #775

Claimed ground (for disjointness checks)

  • Adds (only file touched): tests/unit/test_update_coverage_snapshot.py (new)
  • Subject under test: scripts/update_coverage_snapshot.pyparse_scenarios, classify, SuiteCounts.total, count_scenarios, render_snapshot, update_file, main
  • Modifies: nothing. No production code, no workflow, no docs.

Disjoint from the open testsuite PRs: #774 (scripts/compose-e2e-image.sh), #773 (SSH transport), #771 (Containerfile.runner SHA-256), #768 (offline_boot_steps.py / sudo_rs_steps), #766 (scripts/generate_skill_index.py), #765 (behave environment quarantine gate), #757 (input-remapper QEMU), #751 (screenshot_steps.py). None of them touch scripts/update_coverage_snapshot.py or a file named tests/unit/test_update_coverage_snapshot.py.

Why

.github/workflows/pr-validate.yml:84 runs python3 scripts/update_coverage_snapshot.py --check as a required gate, and AGENTS.md forbids hand-editing the numbers it generates — but the script itself had zero unit tests. The gate is self-referential: it diffs the script's output against a table the same script produced, so a tag-precedence or feature-tag-inheritance regression moves both sides together and CI stays green while the suite-map numbers drift.

What this adds

44 tests in tests/unit/test_update_coverage_snapshot.py:

Area Coverage
parse_scenarios untagged scenarios, scenario tags, feature-tag inheritance, union of feature + scenario tags, tags not leaking to the next scenario, Scenario Outline counted once (not per Examples: row), comments ignored, Rule:/Background:/Examples: reset pending tags, hyphen/underscore tag names, file with no scenarios
classify active default, unknown tags stay active, each of pending/future/hardware_blocked → pending, quarantine outranks every backlog tag
SuiteCounts total sums all three buckets; zero defaults
count_scenarios suite name = first path part under tests/, bucketing by precedence, multi-file accumulation, multi-suite separation, non-.feature files ignored
render_snapshot marker wrapping, totals line (scenario + feature-file counts), rows sorted by suite, per-row column values, SUITE_NOTES prose for known suites, empty note for unknown suites, header row
update_file missing markers → exit 2 with guidance on stderr, missing end marker → exit 2, write mode replaces block / preserves surrounding text / is idempotent, --check returns 1 on stale and never writes, --check returns 0 when current, --check catches a newly added scenario
main --repo-root honoured, --check propagates and does not write
repository invariant the committed snapshot is current — the CI --check assertion, reproduced inside the unit suite

All fixtures build synthetic feature trees under tmp_path; nothing touches the real docs/skills/test-authoring/suite-map/SKILL.md except the read-only invariant test.

Verification

$ python3 -m pytest tests/unit/test_update_coverage_snapshot.py -q
44 passed

Full unit suite before and after this branch is identical — 3 failed, 9 errors in both runs, all pre-existing and caused by behave/webdriver not being installed in the local sandbox; passing count goes 1236 → 1280 (+44).


Filed by quality agent (hold-gated mode). Human review required — do not merge without review.

— hive: agent=quality backend=copilot model=claude-opus-5

scripts/update_coverage_snapshot.py is a required PR gate — pr-validate.yml
runs it with --check to block stale suite-map coverage tables — but it had
zero unit tests. A regression in its tag parsing or classification would
silently change the scenario numbers every test PR is measured against.

Adds tests/unit/test_update_coverage_snapshot.py: 44 tests covering
parse_scenarios (feature-tag inheritance, per-scenario tag scoping,
Scenario Outline counted once, comment lines, Rule:/Background:/Examples:
tag resets), classify (quarantine > hardware_blocked/future/pending >
active precedence), SuiteCounts.total, count_scenarios (suite name derived
from the first path part under tests/, multi-file and multi-suite
accumulation, non-.feature files ignored), render_snapshot (marker
wrapping, totals line, sorted rows, column values, SUITE_NOTES lookup),
update_file (missing-marker exit 2, write/idempotent/check modes, check
never writes) and main (--repo-root and --check plumbing). Also asserts the
committed snapshot is current, so the CI invariant is reproduced locally.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: sec-check[bot] <sec-check[bot]@users.noreply.github.com>
@kubestellar-hive kubestellar-hive Bot added hold Work is intentionally paused. quality testing agent/quality Approved by a Hive merger/owner for auto-merge on green CI hive/hosted-projectbluefin-knuckle-gjvq Approved by a Hive merger/owner for auto-merge on green CI labels Aug 31, 2026
@codecov-commenter

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agent/quality Approved by a Hive merger/owner for auto-merge on green CI hive/hosted-projectbluefin-knuckle-gjvq Approved by a Hive merger/owner for auto-merge on green CI hold Work is intentionally paused. quality testing

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant