fix(lint): see the opencode sandbox, and stop reading "names" as "writes" - #36
Merged
Conversation
…tes" On 2026-08-16 an ohalloran-demonstrator site-build run declared its deliverables inside a Meridian session scratchpad. The opencode Seatbelt profile denies writes there, so both opencode lanes did the work, could not deliver it, exited 0, and were recorded failure_class=model. claude-opus-4.8 and claude-sonnet-5 now read 0% on site-build for a manifest that was unwritable. The lint that exists to catch exactly this stayed silent, for two independent reasons. 1. engine_confines_writes_to_taskdir keyed solely on codex's "workspace-write" vocabulary, calling opencode's boundary "invisible from here". It never was: the Seatbelt profile ships in engines/ and its writable surface is TASKDIR plus the per-run scratch. Recognise the wrappers this repo ships by bin name, and let any other wrapper declare itself with a new engine field, confines_writes_to_taskdir. An engine ringer genuinely cannot read still stays silent — that principle was right and is now tested directly. 2. The check-exports exemption tested whether the check CONTAINED the path. Naming a path is not writing it: this run's check was `python3 check-candidate.py /abs/candidate-c-drawn.html`, which READS the file the worker was meant to produce — the failing case itself. The lint's own docstring already said an opaque bare script call should still warn, so the substring test contradicted the documented design. Exempt only on evidence of writing: a redirect onto the path, or a copy-shaped verb in the same command segment. Half 2 is the one that mattered. Fixing only half 1 would have left this run silent, because its check named the path. Verified by replaying the real task — the recorded check verbatim, the real declared deliverable, the real ~/.config/ringer/config.toml: 0 warnings before, 1 after. Tests cover both halves, the segment boundary (a cp earlier in a check must not vouch for a path named later), explicit declaration in both directions, full_access exemption, and that this stays a WARNING and never blocks a run. test_w12_unconfined_engine_is_quiet is reversed, deliberately: it asserted the silence this change removes. Its comment records why, and the narrower principle it was really protecting is kept in test_an_unknown_engine_bin_stays_silent. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes the gap found while investigating the uncommitted MODEL-NOTES work (#35). Two independent holes, and the second is the one that actually mattered.
What happened at 02:35 this morning
ohalloran-demonstrator, site-build. Both opencode lanes failed; the codex lane passed.The workers exited cleanly. The manifest declared deliverables inside a Meridian session scratchpad, which the opencode Seatbelt profile denies writes to. The models did the work and physically could not deliver it — and
claude-opus-4.8andclaude-sonnet-5now read 0% on site-build for a manifest that was unwritable. A surface misstating what happened, same class aslou-call-transcriptand the ANSI corruption in #33.sandbox_unreachable_deliverable_findingsexists precisely to catch this. It stayed silent for two reasons.Hole 1 — the engine's boundary was called invisible
engine_confines_writes_to_taskdirkeyed only on codex'sworkspace-writevocabulary. Its docstring:It never was invisible. The Seatbelt profile ships in
engines/, a few hundred lines from that function, and its writable surface is exactlyTASKDIR+ the per-run scratch.Now: recognise the wrappers this repo ships by bin name, and let any other wrapper declare itself with a new engine field,
confines_writes_to_taskdir(true or false). An engine ringer genuinely cannot read still stays silent — that principle was right, and is now tested directly rather than being a side effect of the codex-only rule.Hole 2 — "names the path" was read as "creates the path"
This is the one that let the run through. The check was:
The exemption tested
path in task.check. That path IS in the check — as an argument to a read-only verifier. Naming a path is not writing it, and a check that reads the file the worker was supposed to produce is the failing case itself, not evidence of a deliberate export design.The lint's own docstring already said what should happen:
A bare script call carrying the path was getting exempted — the substring test contradicted the documented design. Now the exemption requires evidence of writing: a redirect onto the path, or a copy-shaped verb (
cp/mv/install/tee/rsync/ln) in the same command segment.Fixing only hole 1 would have changed nothing this morning.
Verification
Replayed the real task — recorded check verbatim, real declared deliverable, real
~/.config/ringer/config.toml:Tests cover both halves, the segment boundary (a
cpearlier in a check must not vouch for a path named later by a verifier), explicit declaration in both directions, thefull_accessexemption, and that this remains a warning that never blocks a run.Full suite: 352 pass.
One reversed test, on purpose
test_w12_unconfined_engine_is_quietasserted the silence this change removes. It's flipped, with a comment recording why and when. The narrower principle it was really protecting — never warn from a guess — is preserved intest_an_unknown_engine_bin_stays_silent.🤖 Generated with Claude Code