fix(validate): honest scoped PASS — define the disclosure homes instead of paying to delete caveats

[boshu2]

The contradiction (empirically proven)

A fresh-install smoke found that skills/validate/scripts/validate.py rejects any PASS whose not_checked is nonempty and silently rewrites the verdict to NOT_PROVEN via a validate.integrity finding — while skills/validate/SKILL.md (output contract + step 5), AGENTS.md Closeout, and docs/architecture/operating-loop.md all instruct disclosing checked and not_checked.

Identical honest draft → NOT_PROVEN. Same draft with not_checked emptied → PASS. The tool paid users to delete their scope caveats — the inverse of the product thesis.

Decision: option (b), with the disclosure homes actually defined

PASS stays strict-empty. Every doc surface now defines exactly where each kind of honest scope limit goes, and the tool names those homes in its error instead of failing silently.

Why not (a) — the recommended option

Two independent reasons, one semantic and one structural.

Semantic (the primary reason). The bug is not an over-strict PASS rule; it is a semantic overload of not_checked. not_checked means in-scope acceptance surface this validation did not verify. A verdict is a judgment over acceptance, and the operating contract already says unverified acceptance is NOT_PROVEN. So PASS ∧ not_checked ≠ ∅ is a genuine contradiction, and relaxing it would destroy the only guarantee that makes PASS worth anything: PASS means the whole declared acceptance surface was verified.

What the smoke actually caught is that honest validators had nowhere else to put a declared non-goal — a thing that was never acceptance in the first place — so they dumped it in the acceptance-only field and got punished for it. Fix the overload, not the strictness.

Structural. verdict.v2 is a tri-implementation contract: the Python writer (skills/validate/scripts/validate.py), the Go reader (cli/internal/verdictcheck), and schemas/verdict.v2.schema.json, kept in agreement by the golden corpus in tests/fixtures/verdict-contract/cases/ (23 cases, including invalid-pass-not-checked-nonempty). Option (a) — not_checked entries as {scope, reason} objects, or a parallel not_checked_reasons field — changes the persisted wire shape and would require:

• cli/internal/verdictcheck/verdictcheck.go: NotChecked []string, DisallowUnknownFields(), and validatePass's len(verdict.NotChecked) != 0 (forbidden write scope),
• cli/internal/statusapp/statusapp.go (forbidden),
• schemas/verdict.v2.schema.json (additionalProperties: false, not_checked: {maxItems: 0} on PASS),
• schemas/rpi-report.v1.schema.json and skills/rpi/scripts/run_once.py (forbidden),
• the golden corpus fixtures.

That is a verdict.v3 wire-format change, not a fix — and shipping the Python leg alone would fork the contract and turn every stored PASS into something ao status reports as invalid.

The four homes (now documented everywhere)

Scope limit	Home	Example
A criterion proven by a bounded check	criteria[].reason on that criterion	"proven by the unit suite; the full integration matrix was not replayed"
A declared non-goal / out-of-scope area	the intent source's non-goals, optionally restated as an evidence-backed boundary criterion in criteria	"cli/** is a declared non-goal; the diff proves it untouched"
Residual risk or judgment caveat	the caller-facing report	"the migration path is untested against pre-3.0 stores"
Acceptance that genuinely went unverified	not_checked — and the result is NOT_PROVEN, not PASS	"criterion 3 needs hardware this context cannot reach"

Note the second row: converting a non-goal into an evidence-backed boundary criterion is stronger than an unexplained not_checked entry — the caveat becomes a proven boundary claim that stays visible in the stored artifact. Nothing is deleted to earn a PASS.

Proof of the incentive fix (the smoke's exact scenario)

Both drafts carry the same honest content. Driven through the real store-verdict CLI:

### DRAFT A - declared non-goals parked in not_checked (the smoke scenario)
stored verdict : NOT_PROVEN
finding        : PASS cannot contain not_checked items: not_checked lists unverified in-scope
                 acceptance surface, so a PASS has none by construction; record a bounded proof
                 of a criterion in criteria[].reason, a declared non-goal in the intent source's
                 non-goals (optionally as an evidence-backed boundary criterion), and residual
                 risk in the report; keep a not_checked entry only when acceptance genuinely
                 went unverified, which is NOT_PROVEN

### DRAFT B - same honest content, caveats in the documented homes
stored verdict : PASS
not_checked    : []
caveats preserved in the stored PASS artifact:
  - c1: proven by the unit suite; the full integration matrix was not replayed
  - non-goal:cli-untouched: declared non-goal; the diff proves cli/** untouched
  - non-goal:windows-runners: declared non-goal in the intent source; not part of acceptance

Before: silent rewrite, and the only way to a PASS was deleting the caveats. After: an actionable message naming where each caveat belongs, and a PASS artifact that still contains all three caveats.

Regression test

Added to the validate skill's existing test surface (skills/validate/scripts/test_validate.py, run by skills/validate/scripts/validate.sh → tests/skills/run-all.sh):

• test_honest_scoped_pass_round_trips_through_documented_homes — draft A (non-goals in not_checked) stores NOT_PROVEN with a finding naming criteria[].reason, non-goals, and the report; draft B (same content, documented homes) stores PASS, schema-valid, with both caveat strings re-read off disk from the persisted artifact.
• test_criteria_field_error_names_the_allowed_set — the store-verdict error names {id, result, evidence_refs, reason} and the offending key.

$ cd skills/validate/scripts && python3 -m unittest -v test_validate
test_criteria_field_error_names_the_allowed_set ... ok
test_honest_scoped_pass_round_trips_through_documented_homes
An honest draft with declared non-goals is representable as PASS. ... ok
[16 pre-existing tests] ... ok
----------------------------------------------------------------------
Ran 18 tests in 0.066s

OK

Also in scope (three verified defects)

1. Install-agnostic helper paths. python3 skills/validate/scripts/validate.py only resolves in a repository checkout; the installed path is .agents/skills/validate/scripts/validate.py. SKILL.md now uses $SKILL_DIR — this SKILL.md's own directory — and names both layouts.
2. manifest documented flagless but requires --root and at least one repeatable --include. A new Helper commands table documents the real required/optional flags for all five subcommands (manifest, verify-manifest, snapshot-intent, digest, store-verdict).
3. store-verdict criteria error now reads verdict.v2 criteria[0] has invalid fields (unknown confidence); allowed fields are {id, result, evidence_refs, reason}, of which id, result, evidence_refs are required.

Surfaces changed (all agree)

• skills/validate/scripts/validate.py — NOT_CHECKED_HOMES constant + contract comment; both PASS/not_checked messages; criteria-field error.
• skills/validate/SKILL.md — new Scope disclosure and Helper commands sections; steps 1, 4, 5, 6 corrected.
• AGENTS.md Closeout (CLAUDE.md symlinks to it — edited AGENTS.md, not the symlink).
• docs/architecture/operating-loop.md — Fresh Validate section.
• Regenerated projections: skills-codex/validate/**, images/gemini/skills/validate/SKILL.md, skills-codex/.agentops-manifest.json, skill mesh.
• Untouched by design: cli/**, skills/plan/**, skills/rpi/**, schemas/verdict.v2.schema.json, tests/fixtures/verdict-contract/**, packs/agentops-executor/** (frozen historical bytes per scripts/check-gc-maintainer-ops.sh).

Gates

bash scripts/regen-all.sh --check       → All generated projections are current.  (exit 0)
cd cli && go build ./...                → exit 0
cd cli && go test ./...                 → exit 0 (66 packages ok, 0 FAIL)
bash tests/skills/run-all.sh            → exit 0 (54 skills passed, 0 failed; 106 checks)
bash scripts/check-verdict-contract-corpus.sh → PASS (schema + python + go all ran, 23 cases)
bash tests/docs/validate-links.sh       → 427 links checked, 0 broken
bash tests/spec-consistency/test-gate.sh → 12 PASS, 0 FAIL
ruff check skills/validate/scripts/     → All checks passed!

Residual risk

• The tri-implementation contract is unchanged, so no stored verdict changes meaning and no reader forks. The only behavioral change is message text plus a marginally more precise criteria-shape check (same accept/reject set — the golden corpus proves it).
• not_checked remains a flat string array. If a future caller genuinely needs machine-readable justified non-goals inside a verdict, that is a deliberate verdict.v3 with all three implementations moved together — not something to smuggle in through the Python writer.