Skip to content

[quality] scripts/update_coverage_snapshot.py is a required PR gate with zero unit tests #775

Description

@kubestellar-hive

Finding

.github/workflows/pr-validate.yml:84 runs python3 scripts/update_coverage_snapshot.py --check as a PR gate. The script (186 lines) is the sole authority for the suite-map Coverage snapshot table — AGENTS.md and docs/skills/ci-ops/ops/references/coverage-snapshot-generated.md both say those numbers must never be hand-edited.

It has no unit tests. Every function is uncovered:

  • parse_scenarios — feature-tag inheritance, per-scenario tag scoping, Scenario Outline counted once, comment lines, Rule:/Background:/Examples: tag resets
  • classify — tag precedence quarantine > hardware_blocked/future/pending > active
  • count_scenarios — suite name derived from the first path part under tests/
  • render_snapshot — totals line, row ordering, column values, SUITE_NOTES lookup
  • update_file — missing-marker exit 2, write vs --check modes
  • main--repo-root / --check plumbing

A regression in tag parsing or precedence would not fail CI: the gate compares the script's output against a table the same script generated, so both sides move together. The numbers would just silently drift, and every test PR is measured against them.

Recommendation

Add tests/unit/test_update_coverage_snapshot.py pinning the parsing and classification contract, plus a repository invariant test that reproduces the CI --check assertion inside the unit suite.

Priority

  • Impact: high (required gate, self-referential — cannot fail on its own regression)
  • Effort: low (pure functions, no I/O beyond tmp_path)

PR: this issue is addressed by the hold-gated PR that adds tests/unit/test_update_coverage_snapshot.py.


Filed by quality agent (hold-gated mode)

🐝 Hive Agent: quality | Instance: hosted-projectbluefin-knuckle-gjvq | SHA: 083ed2b4

— hive: agent=quality backend=copilot model=claude-opus-5

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions