Skip to content

feat(shots): before/after PR evidence kit - #72

Merged
changeroa merged 1 commit into
mainfrom
feat/audit-p4-before-after
Oct 19, 2026
Merged

changeroa merged 1 commit into
mainfrom
feat/audit-p4-before-after

Conversation

@changeroa

@changeroa changeroa commented Oct 19, 2026 •

Copy link
Copy Markdown
Owner

Summary

Adds P4, the before/after PR evidence kit. stylegallery shots --before <ref|url> --after <ref|url> captures 390 and 1440 widths x light/dark (x focus/hover when --selector is given) for both sides with Playwright Chromium, computes a pixel diff ratio, a bounding box of changed pixels, and visual-qa grid hotspots per pair, and writes before/, after/, diff/ PNGs, report.json, and pr-body.md (a before|after table per viewport, a violation-id slot, verification results, and the evidence boundary sentence). A side is a Git ref (exported with git archive and served locally), . for the working tree, or an http(s)/file URL. PNG decoding and diffing run inside Chromium, so no new dependency is added. Plain stylegallery shots without --before/--after still routes to the site compiler unchanged.

Also adds the governed how-to quality/evidence/before-after-pr-evidence.md, a copyable PR body template inside that doc, README and CLI help usage, and scripts/test-shots-compare.mjs (bun run test:shots), which proves identical pages diff to 0 and an injected 200x120 block yields a nonzero ratio with the exact bbox 40,300 200x120, across Git refs, the working tree, URLs, and argument errors. CI runs it in the compiler workflow after installing Chromium.

Governance Classification

  • PR template placement: the template lives in the governed doc rather than .github/pull_request_template.md. scripts/validate-okf.mjs requires OKF frontmatter on every Markdown file outside .git/.omo/node_modules, and GitHub would insert that frontmatter into every new PR body. Exempting .github would be a validator scope change for the Validation owner, so it is left as a follow-up.
  • Shared infrastructure, Quality owner: quality/evidence/before-after-pr-evidence.md is an Evidence Reference beside visual-qa.md, routed from quality/evidence/index.md and admitted to material v2 as shared/stable like its neighbors (quality/** family in GOVERNANCE.md). No domain guidance, Layout pattern, token, or profile value changes.
  • CLI transport: scripts/shots-compare.mjs and the shots dispatch in scripts/stylegallery-cli.mjs; the compiler adapter is untouched.
  • Generated or sealed artifacts updated by the repository's own generators: consumer-reference/agent-native/v2/material-registry.json, consumer-reference/agent-native/v2/search-embeddings.json (197 documents), the admission policy version_id, the admitted-count pins 196 -> 197 (on top of Add audit ledger schema and stylegallery audit status (P5) #71, feat(expression): AI-slop UI anti-pattern catalog (P3) #70, and Add HIG universal principles extraction (P1) #73) (schema and material tests), and the a2a/ag-ui lifecycle inventory hashes and counts via generateProtocolExtensionInventory (no disposition, owner, deadline, or caller-status change).
  • Search eval: questions q63 (must-pass), q64, and m22 for the new doc.
  • No version bump and no log.md entry; release is separate.

Consumer Reference

consumer_reference: consumer-reference/agent-native/registry.json

Before/After

UI is unchanged, so this shows CLI output on origin/main (1836913, before #70, #71, #73) vs this branch.

stylegallery shots --before origin/main --after ., before:

{
  "ok": false,
  "operation": "shots",
  "failures": [
    {
      "code": "argument_invalid",
      "message": "Unknown or repeated argument: --before"
    }
  ]
}

After (branch), stylegallery shots --before origin/main --after . --page showcase/index.html --selector "main a": ok: true, 12 pairs (390/1440 x light/dark x default/focus/hover), 0 changed, max ratio 0 (Chromium 149.0.7827.55). The ratio is 0 because this PR changes no page. It writes report.json and pr-body.md.

stylegallery help, diff from origin/main to this branch:

38a39,52
> Before/after PR evidence (local Chromium; no site compiler needed):
>   shots --before <ref|url> --after <ref|url>
>                                 Capture 390 and 1440 widths x light/dark (x focus/hover with --selector)
>                                 for both sides, diff each pair, and write before/, after/, diff/,
>                                 report.json, and pr-body.md. A side is an http(s) or file URL, a Git
>                                 ref, or . for the working tree.
>       --page <path>             Repository-relative page served for a ref or . (default index.html).
>       --out <dir>               Output directory (default stylegallery-shots-compare in the system temp dir).
>       --selector <css>          Element to focus and hover for the extra states.
>       --threshold <0-1>         Per-channel tolerance before a pixel counts as changed (default 0.05).
>       --full-page               Capture the full page instead of the viewport.
>       --wait <ms>               Extra settle time after load (default 0).
>       --violations <ids>        Comma-separated audit violation ids for the PR body.
>

Violation IDs

None. This adds tooling and fixes no audit violation.

Verification

All run locally on the tree rebased onto 9830abf (#71, #70, #73):

  • Every command in .github/workflows/validate.yml (validate job, 102 commands, including generated-artifact drift, OKF, links, IA, vocabulary, governance, domains, consumer-reference, agent-native, material v2, lifecycle dispositions, and pattern checks): all exit 0.
  • bun run validate: exit 0. bun run test: exit 0.
  • bun run eval:search (embeddings --check + hybrid gate) and node scripts/search-eval.mjs --gate --mode lexical: pass.
  • node --test scripts/test-stylegallery-search.mjs, node --test scripts/test-stylegallery-setup.mjs, bun run test:compiler: pass.
  • bun run test:audit (from Add audit ledger schema and stylegallery audit status (P5) #71): pass.
  • bun run test:shots: 4/4 pass (identical pages ratio 0 with bbox null; injected block ratio > 0 with bbox 40,300 200x120 and diffPixels 24000 in all 12 pairs; Git ref + working tree; argument errors).

Screenshot diffs show rendered change, not usability.

@changeroa
changeroa force-pushed the feat/audit-p4-before-after branch 2 times, most recently from c78d5ff to b2383d2 Compare October 19, 2026 12:10
…e/--after)

Capture 390/1440 x light/dark (x focus/hover) for two revisions, diff each pair in Chromium, and write PNGs, an image-diff-compatible report.json, and pr-body.md. Add the Before/After PR Evidence doc with a PR body template, README/CLI help, search-eval questions, and a self-test run in the compiler workflow.
@changeroa
changeroa force-pushed the feat/audit-p4-before-after branch from b2383d2 to af97cb4 Compare October 19, 2026 12:30
@changeroa
changeroa merged commit 393c2d5 into main Oct 19, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant