Skip to content

fix(models): keep a worker's model banner when its output outruns the buffer - #29

Merged
usjoh merged 1 commit into
mainfrom
fix/model-attribution-head-capture
Aug 1, 2026
Merged

fix(models): keep a worker's model banner when its output outruns the buffer#29
usjoh merged 1 commit into
mainfrom
fix/model-attribution-head-capture

Conversation

@usjoh

@usjoh usjoh commented Jul 31, 2026

Copy link
Copy Markdown
Owner

What was wrong

Model attribution degraded silently with run length.

Harnesses announce their model in a banner at startup, but reported_model was scraped at exit from a 1MB rolling tail (RollingBytes(max_bytes=1_000_000)). Any worker that talked past 1MB scrolled its own identity out of the buffer, and nothing downstream could distinguish that from a harness that never announced itself.

That stayed invisible because stamped_model = reported_model or resolved_model falls back to config. But the built-in codex template uses the optional {model_args} placeholder — with no model pinned it drops the flag entirely, so resolved_model is empty too and the attempt lands under (unattributed legacy rows) permanently. The default in force at run time is recorded nowhere and cannot be reconstructed afterwards.

Both conditions must hold, which is why the same orchestrator produced both attributed and blank rows and the gap looked intermittent.

Evidence

Found while reviewing this machine's own usage, where 16 unattributed rows had appeared since 2026-07-29 despite the gap being closed in July. Among observer-triad attempts the split is exact:

over 1MB output
unattributed rows 15 / 15
attributed rows 3 / 17

Smallest blank row: 1,074,767 bytes — 75KB past the cap.

The fix

  • RollingBytes retains a bounded head (64KB) alongside the tail; the scrape falls back to it.
  • Tail still wins, so every run that resolves today resolves identically — the head only fills in an answer that was previously lost. Token counts need no fallback; harnesses report those at the end, inside the surviving window.
  • New lint warning when an optional-{model_args} engine has neither a task model nor a model_default. A warning, not an ERROR: — the run is legitimate, only its provenance is lost, so run still proceeds. The required-{model} form already raises in validate_manifest_engines and is not double-reported.

Testing

317 → 325 tests, full suite green.

Both fixes are mutation-tested at the call site, which is the part that matters here: scraping the head in isolation, or building the head buffer without wiring it into _run_worker, passes a unit-level test just as well while changing nothing. Three mutations applied, three caught:

  • call-site head fallback removed → fails
  • head_bytes not passed at the capture site → fails
  • unpinned_model_findings not wired into lint_manifest → fails

Verified against the real manifests that triggered this: they linted clean before and now report the unpinned model on every task.

… buffer

Attribution silently degraded with run length. Harnesses announce their
model in a banner at startup, but reported_model was scraped from a 1MB
ROLLING TAIL at exit, so any worker that talked past 1MB scrolled its own
identity out of the buffer before anyone read it. Nothing downstream could
tell that apart from a harness that never announced itself.

That was survivable while stamped_model could fall back to the config, but
the built-in codex template uses the OPTIONAL {model_args} placeholder: with
no model pinned it drops the flag entirely, resolved_model is empty too, and
the attempt lands under "(unattributed legacy rows)" forever — the default in
force at run time is recorded nowhere and cannot be reconstructed after.

Both conditions have to hold, which is why the same orchestrator produced
both attributed and blank rows and the gap read as intermittent. In this
machine's log the split is exact: of the observer-triad attempts, 15/15 of
the blank ones exceeded 1MB of worker output against 3/17 of the attributed
ones, the smallest blank sitting 75KB past the cap.

RollingBytes now retains a bounded head alongside the tail, and the scrape
falls back to it. Tail still wins, so every run that resolves today resolves
identically and the head only ever fills in an answer that was previously
lost. Token counts need no fallback — harnesses report those at the end.

Also warn at lint time when an optional-{model_args} engine has neither a
task "model" nor a model_default: the run is legitimate so it is a warning
rather than an ERROR, but the author now hears about it before the evidence
is gone instead of never. The required-{model} form already raises in
validate_manifest_engines and is not double-reported.

Both fixes are mutation-tested at the call site — scraping the head in
isolation, or building the head buffer without wiring it up, passes a
unit-level test just as well while changing nothing.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@usjoh
usjoh merged commit 08654bf into main Aug 1, 2026
2 of 3 checks passed
@usjoh
usjoh deleted the fix/model-attribution-head-capture branch August 1, 2026 10:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant