fix: Strip ANSI escapes before parsing token counts and reported models - #99
Open
chiv-heng wants to merge 1 commit into
Open
fix: Strip ANSI escapes before parsing token counts and reported models#99chiv-heng wants to merge 1 commit into
chiv-heng wants to merge 1 commit into
Conversation
- When an engine CLI is launched with FORCE_COLOR in the environment (as Claude Code and most CI systems set), codex colorizes its stdout even through a pipe: the captured bytes are "\x1b[1mmodel:\x1b[0m gpt-5.6-sol" and "\x1b[2mtokens used\x1b[0m" with the number on the next line. The shipped DEFAULT_TOKEN_REGEX and DEFAULT_CODEX_MODEL_REPORT_REGEX never match those bytes, so every run silently records no model and no tokens. - Fix the class, not the instances: parse_token_count and parse_reported_model strip ANSI with the existing ANSI_RE (already used by clean_log_text for display) before matching. Both shipped regex constants are untouched and user-supplied custom regexes benefit equally. - Verified by a controlled A/B run: identical one-task manifest, stock config, codex-cli 0.146.0 — with FORCE_COLOR=3 the eval row logged tokens=None model=None; with it unset, tokens=17238 model=gpt-5.6-sol. Colorized output confirmed on codex 0.144.1 and 0.146.0 alike; the trigger is the launch environment, not the codex version. AI-assisted: Yes (Claude Code) AI contribution: root-cause investigation (A/B probe under ringer), patch, regression tests Human review: Yes – I reviewed, tested, and approved this change. Claude-Session: https://claude.ai/code/session_01RNCh1cr6wJHXj52Hwwi5KW
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Strip ANSI escapes before parsing token counts and reported models
The observed failure
Every codex run launched from Claude Code on my machine recorded no model and no tokens — the scoreboard attributed nothing while the raw worker log plainly showed both. Wrong model attribution is the bug class this repo's CONTRIBUTING calls unforgivable, which is why I'm sending this one upstream.
Root cause: environments like Claude Code and most CI systems set
FORCE_COLORfor every child process, ringer passes its environment to workers unchanged, and codex honors the variable — it colorizes stdout even through a pipe. The captured bytes become:DEFAULT_CODEX_MODEL_REPORT_REGEXanchors on^model:and never matches the bold-wrapped header.DEFAULT_TOKEN_REGEX's\s*cannot bridge the escape bytes between "used" and the number. Both parse toNone, silently.Executed proof
Controlled A/B under ringer itself — identical one-task manifest, stock
[engines.codex]from config.sample.toml, codex-cli 0.146.0, one variable flipped:FORCE_COLOR=3(Claude Code default)worker_tokens: null,reported_model: nullenv -u FORCE_COLOR, all else identicalworker_tokens: 17238,reported_model: "gpt-5.6-sol"Version is not the trigger: captured worker logs show codex 0.144.1 and 0.146.0 colorizing byte-identically under
FORCE_COLOR, and 0.145.0 parsing fine without it. Any engine CLI that honorsFORCE_COLOR/--color=alwaysconventions will reproduce this; uncolorized environments are unaffected (stripping is a no-op on plain text).The fix
Fix the class, not the instances:
parse_token_countandparse_reported_modelstrip ANSI once with the module's existingANSI_RE— already the accepted remedy in this codebase (clean_log_textuses it for log display) — before matching. Both shipped regex constants are untouched, and user-written customtoken_regex/model_report_regexoverrides benefit equally, which patching the two constants would not achieve.7 lines in
ringer.py(2 functional + comments), no new constants, stdlib only.Tests
tests/test_ringer.py: colorized token output (real captured bytes) parses via the shipped default and via a user-style custom regex; uncolorized input still parses (no regression).tests/test_identity_evidence.py: bold-wrappedmodel:header parses via the default codex engine regex, alongside the existing plain-header test.Both new tests fail on unpatched
ringer.py(None), pass with the fix. Full suite: 255 tests, all green, macOS / Python 3.14 and 3.12.🤖 Generated with Claude Code
https://claude.ai/code/session_01RNCh1cr6wJHXj52Hwwi5KW