Skip to content

ci: add coverage gate and benchmark job, bump action pins - #6

Open
lbliii wants to merge 1 commit into
mainfrom
ci/coverage-gate-bench-pins
Open

ci: add coverage gate and benchmark job, bump action pins#6
lbliii wants to merge 1 commit into
mainfrom
ci/coverage-gate-bench-pins

Conversation

@lbliii

@lbliii lbliii commented Jul 8, 2026

Copy link
Copy Markdown
Owner

Implements three tasks of #5 (hygiene baseline alignment): a coverage gate, a CI benchmark job, and current action pins.

Coverage gate

  • Measured on Python 3.14t (free-threaded) after regenerating fixtures the same way the CI test job does:
    uv run python scripts/generate_fixtures.py --update
    uv run pytest -q --tb=short --cov=rosettes --cov-report=xml
    
    Result: 79.30% (7863/9915 lines, 2026-07-08).
  • Added [tool.coverage.report] fail_under = 79 to pyproject.toml as an honest ratchet toward 80. pytest-cov picks this up from the existing --cov invocation in the test job — verified locally: Required test coverage of 79.0% reached. Total coverage: 79.30%.

Benchmark job (non-gating)

New benchmarks job in tests.yml:

  • Installs pytest-benchmark + pygments at the job level (kept out of the dev group / lockfile since they're only needed here).
  • Runs uv run --no-sync pytest benchmarks/benchmark_vs_pygments.py --benchmark-only --benchmark-json=benchmark-results.json (the file must be passed explicitly — python_files = ["test_*.py"] excludes benchmark_*.py from directory collection). 17 benchmarks incl. the Pygments comparison and the slow 10k-line cases; ~22s locally.
  • Also runs benchmarks/benchmark_parallel.py for free-threading scaling numbers.
  • Uploads benchmark-results.json as an artifact. No baseline comparison yet — that's the evidence epic (Epic: Evidence — prove the ReDoS-safety and speed claims (P1) #4).

Action pins

Bumped to ecosystem current (matching lbliii/kida) across all four workflows:

Action Old New
actions/checkout v4 v7
astral-sh/setup-uv v4 v7
actions/cache v4 v6
actions/setup-python v5 v6
actions/upload-artifact v4 v7
actions/download-artifact v4 v8
actions/upload-pages-artifact v3 v5
actions/deploy-pages v4 v5

pypa/gh-action-pypi-publish@release/v1 left as-is (floating release branch).

Verification

  • Full test suite green on 3.14t with the gate active.
  • Benchmark invocation and JSON artifact verified locally; parallel script runs clean.
  • All workflow YAML validates.

Part of #5

🤖 Generated with Claude Code

- Enforce coverage floor via [tool.coverage.report] fail_under = 79
  (honest ratchet toward 80; measured 79.30% on 2026-07-08). The existing
  test job's --cov run now fails below the floor via pytest-cov.
- Add a non-gating benchmarks CI job: runs the pytest-benchmark suite
  (benchmark_vs_pygments.py, incl. Pygments comparison) and the parallel
  scaling script, uploading benchmark-results.json as an artifact.
  Baseline comparison is deferred to the evidence epic.
- Bump action pins to ecosystem current (matching lbliii/kida):
  checkout v4->v7, setup-uv v4->v7, cache v4->v6, setup-python v5->v6,
  upload-artifact v4->v7, download-artifact v4->v8,
  upload-pages-artifact v3->v5, deploy-pages v4->v5.

Part of #5

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings July 8, 2026 15:47

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants