fix(models): keep a worker's model banner when its output outruns the buffer - #29
Merged
Merged
Conversation
… buffer
Attribution silently degraded with run length. Harnesses announce their
model in a banner at startup, but reported_model was scraped from a 1MB
ROLLING TAIL at exit, so any worker that talked past 1MB scrolled its own
identity out of the buffer before anyone read it. Nothing downstream could
tell that apart from a harness that never announced itself.
That was survivable while stamped_model could fall back to the config, but
the built-in codex template uses the OPTIONAL {model_args} placeholder: with
no model pinned it drops the flag entirely, resolved_model is empty too, and
the attempt lands under "(unattributed legacy rows)" forever — the default in
force at run time is recorded nowhere and cannot be reconstructed after.
Both conditions have to hold, which is why the same orchestrator produced
both attributed and blank rows and the gap read as intermittent. In this
machine's log the split is exact: of the observer-triad attempts, 15/15 of
the blank ones exceeded 1MB of worker output against 3/17 of the attributed
ones, the smallest blank sitting 75KB past the cap.
RollingBytes now retains a bounded head alongside the tail, and the scrape
falls back to it. Tail still wins, so every run that resolves today resolves
identically and the head only ever fills in an answer that was previously
lost. Token counts need no fallback — harnesses report those at the end.
Also warn at lint time when an optional-{model_args} engine has neither a
task "model" nor a model_default: the run is legitimate so it is a warning
rather than an ERROR, but the author now hears about it before the evidence
is gone instead of never. The required-{model} form already raises in
validate_manifest_engines and is not double-reported.
Both fixes are mutation-tested at the call site — scraping the head in
isolation, or building the head buffer without wiring it up, passes a
unit-level test just as well while changing nothing.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What was wrong
Model attribution degraded silently with run length.
Harnesses announce their model in a banner at startup, but
reported_modelwas scraped at exit from a 1MB rolling tail (RollingBytes(max_bytes=1_000_000)). Any worker that talked past 1MB scrolled its own identity out of the buffer, and nothing downstream could distinguish that from a harness that never announced itself.That stayed invisible because
stamped_model = reported_model or resolved_modelfalls back to config. But the built-in codex template uses the optional{model_args}placeholder — with no model pinned it drops the flag entirely, soresolved_modelis empty too and the attempt lands under(unattributed legacy rows)permanently. The default in force at run time is recorded nowhere and cannot be reconstructed afterwards.Both conditions must hold, which is why the same orchestrator produced both attributed and blank rows and the gap looked intermittent.
Evidence
Found while reviewing this machine's own usage, where 16 unattributed rows had appeared since 2026-07-29 despite the gap being closed in July. Among
observer-triadattempts the split is exact:Smallest blank row: 1,074,767 bytes — 75KB past the cap.
The fix
RollingBytesretains a bounded head (64KB) alongside the tail; the scrape falls back to it.{model_args}engine has neither a taskmodelnor amodel_default. A warning, not anERROR:— the run is legitimate, only its provenance is lost, sorunstill proceeds. The required-{model}form already raises invalidate_manifest_enginesand is not double-reported.Testing
317 → 325 tests, full suite green.
Both fixes are mutation-tested at the call site, which is the part that matters here: scraping the head in isolation, or building the head buffer without wiring it into
_run_worker, passes a unit-level test just as well while changing nothing. Three mutations applied, three caught:head_bytesnot passed at the capture site → failsunpinned_model_findingsnot wired intolint_manifest→ failsVerified against the real manifests that triggered this: they linted clean before and now report the unpinned model on every task.