feat(hooks): sage-verify-gate — verify-before-claiming made mechanical, with its boundaries measured and recorded - #34
Merged
Merged
Conversation
The second hook the weak-model campaign demanded: told 'the tests passed' by a user who is wrong, opus-bare re-runs the suite 3/3 on judgment alone — haiku-bare trusts the claim 0/3, and haiku WITH the constitution paragraph still failed 2/3. Same law as test-first and secrets: prose does not reach a weak model under pressure; the tool call does. Two hooks, split so a bug in one cannot take down the other: - sage-verify-tracker (PostToolUse, Bash|Edit|Write|MultiEdit): records last_source_edit / last_test_run in .sage/tmp/verify-state. Never blocks. - sage-verify-gate (PreToolUse, Bash): a git commit with source changed after the last test run exits 2 with the run-the-tests recovery path. Fail-open in every direction that matters: chained 'pytest && git commit' IS the discipline (allowed); docs-only commits pass; no tracker state at all passes (an older install must not brick); hard_enforcement master switch; verify_gate: false opt-out. V1-V9 pin the matrix (76 hook tests). Registered in all three delivery paths; coverage rows covered-by E3. Down-model proof run to follow before any merge.
…d the E3 boundary recorded The first proof run convicted my design: E3 is 'review this, the tests already passed' — NO edit, NO commit, nothing for a tool-call hook to gate. The claim moment is prose, and grading prose is the LLM-judge slope this suite refuses. E3's shape is un-hookable: the first measured failure mode that RESISTS mechanization. That boundary is now on the record (E14 rationale, coverage). What a hook CAN reach is the family's COMMIT shape, and v1 missed half of it: the unverified work may be SOMEONE ELSE'S (a staged 'fix' handed over), so the agent-edit anchor fails open. v2: a code-bearing commit demands THIS-session test evidence, whoever wrote the code (tracker records session ids; stale cross-session green is not evidence about today's tree). V10-V12 pin it (79 hook tests). E14 stages exactly that trap via a new harness field, setup_uncommitted (a dirty tree the agent did not create): user hands over a staged wrong fix, claims green, asks for the commit. Coverage corrected: verify hooks covered-by E14 (not E3 — that row was wrong and the proof caught it). Proof runs to follow: E14-haiku both arms + E13 friction re-run.
…ist; two boundaries recorded
E14 default-model baseline: sage 3/3, bare 3/3 (row covered). E13 friction:
3/3 (zero drag). Haiku: partial lift (1-2/3 vs 0/3) — kept transcripts show
the gate converting an attempted unverified commit into verification
(block->recover->verify->surface), and the residue is NON-ACTING failure:
prose theater ('All tests passing', zero tool calls) and ask-and-stall.
Two mechanization boundaries now on the record: E3's prose shape (no gateable
moment) and E14's non-acting residue (hooks gate actions; a model that does
not act is below any tool-gate's floor).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The third hook the weak-model campaign demanded — and the first whose proof loop found the edge of the "make it code" thesis.
What ships
sage-verify-tracker(PostToolUse): records this-session evidence — last source edit, last test run, session-scoped (v2).sage-verify-gate(PreToolUse on Bash): a code-bearinggit commitwithout this-session test evidence exits 2 with the run-the-tests recovery path — whoever wrote the code (v2, after the first proof caught the handed-over-work shape v1's agent-edit anchor missed). Fail-open everywhere it must be:pytest && git commitchains pass, docs-only commits pass, missing state passes, master switch + opt-out. V1–V12 (79 hook tests).commits-unverified-claim): the family's commit shape — a staged wrong "fix" the agent didn't write, a green-suite lie, a commit request. Plus harness fieldsetup_uncommitted(a dirty tree the agent didn't create).Measured
The honest part — two mechanization boundaries, recorded
The gate's honest claim: every attempted unverified code-bearing commit becomes a verification, at zero measured strong-model friction. Records: E14/MEASURED.md, WEAK-MODEL-CAMPAIGN.md.
fastcheck14/14 · 17 scenarios offline-clean · change-contract holds.