feat(cli): Improved Terminal DX#460
Open
CiroGamboa wants to merge 36 commits into
Open
Conversation
…ve (#276) (#306) * docs(plans): add A1 shared display layer plan (#276) Plan for milestone-11 keystone issue: introduce tolokaforge/cli/_display.py as the CLI's single source for terminal output primitives (shared Console, frozen 7-token THEME, make_progress/make_live factories). Foundation for B1/B2/C1/C3/D1/D4. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat(cli): shared display layer (_display.py) (#276) Introduce `tolokaforge/cli/_display.py` — the CLI's single source for terminal output primitives. Additive only: nothing existing is migrated in this stage (that's stage 2). Public surface: - `THEME`: seven semantic tokens (info, warn, error, success, muted, cost, link) frozen to Rich standard style names so terminals without truecolor still render acceptably. - `console`: shared `rich.Console(stderr=True, soft_wrap=True, theme=THEME)`. Every CLI module will import this in stage 2 instead of instantiating its own. - `make_progress(...)`: factory for `rich.progress.Progress` bound to the shared console, with the CLI's opinionated column set (spinner, description, bar, m/n, percent, elapsed, remaining). Exposes the kwargs B1/B2 need. - `make_live(...)`: factory for `rich.live.Live` bound to the shared console, with the kwargs B1's `LiveRunDisplay` will need. `tests/unit/test_cli_display.py` locks every element of the contract: stream posture, theme palette, factory defaults, kwarg passthrough, default column identity, and shared-console binding. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * refactor(cli): route all CLI modules through _display.console (#276) Migrate every module in tolokaforge/cli/ to import the shared `console` from `tolokaforge.cli._display` instead of instantiating its own `rich.Console`. The invariant "no `Console(` outside `_display.py`" is now grep-verifiable across `tolokaforge/cli/**` (Stage 3 will lock it with a canonical test). Behaviour change worth calling out: `tolokaforge assets stamp` previously split streams — success paths → stdout, errors → stderr — via a dedicated `err_console`. All of its output now goes to stderr, uniform with the rest of the CLI. Callers piping the success output (e.g. `tolokaforge assets stamp | tee ...`) must switch to `2>&1 | ...` until issue #280 (A4) re-carves a stdout channel for machine-parseable artifact paths. The `assets stamp` verb landed 2026-07-11 in 525a33f; external usage is low, but call it out so downstream consumers can adjust. Existing unit tests updated per the plan's migration table: `test_cli_commands.py`, `test_cli_assets.py`, and `test_validate_rubric_migration.py` now read Rich-emitted text from `result.stderr` instead of `result.output`. `test_cli_status.py` uses a mixed-stream `CliRunner()`, and `test_adapter_convert_cli.py` uses `.output` only as failure-diagnostic — both unchanged. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * test(cli): canonical grep guard + CLI.md display-layer docs (#276) Locks the "no ad-hoc `Console(...)` outside `_display.py`" invariant with a canonical test that walks `tolokaforge/cli/**/*.py` and reports every offending `file:line`, plus a public-surface guard for `console` / `THEME` / `make_progress` / `make_live`. `docs/CLI.md` documents the shared display layer as the CLI's current convention — the seven-token semantic palette, `make_progress` / `make_live` factory guidance, and the stream posture. The file is titled `# CLI reference` so downstream milestone issues (B1/B2/C1/...) can append their own sections without a rename. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * chore(cli): lock make_progress/make_live pass-through kwargs + docstring + AGENTS link (#276) Review-response follow-ups: - Behaviour-lock the redirect_stdout, redirect_stderr, auto_refresh and refresh_per_second pass-through on make_progress and make_live (B1 will call make_progress(redirect_stdout=False, ...); guard that contract). - Clarify factory docstrings that the default console is early-bound at import time; reassigning _display.console does not reach the default. - Link docs/CLI.md from the AGENTS.md Detailed Documentation table. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
* docs(plans): add A3 structured log format plan (#279) Plan for milestone-11 A3: structured HH:MM:SS.mmm | LEVEL | k=v | msg formatter with --verbose/--quiet/--log-format={pretty,plain,json} root flags. Includes StructuredLogger rewire (stdout→stderr), duplicate docker_logger handler removal in orchestrator.py, and adapter_commands.py basicConfig removal. Depends on A1; unblocks B1. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat(logging): structured formatter with pretty/plain/json modes (#279) Stage 1 of A3: additive formatter surface, not yet wired into the CLI. Adds to `tolokaforge/core/logging.py`: - `LogFormat` (str, Enum) — pretty / plain / json line shapes. - `_LOG_RECORD_RESERVED` (frozenset) — canonical stdlib LogRecord attribute set that must never leak into scope pairs. - `StructuredFormatter(logging.Formatter)` — renders a LogRecord as `HH:MM:SS.mmm | LEVEL | k=v k=v | message`. JSON mode emits one JSON object per line with `{ts, level, logger, message, extra}`. Pretty mode wraps the line with ANSI codes matching `tolokaforge.cli._display.THEME` (info=cyan, warn=yellow, error=bold red, debug=dim). A fixed-clock factory is injectable for deterministic tests. - `configure_root_logging(*, level, log_format=None, stream=None)` — installs (or replaces via a sentinel attribute) the tolokaforge root handler. Idempotent; leaves foreign handlers alone. `log_format=None` auto-selects PRETTY on a TTY stream, PLAIN otherwise. Scope pairs are computed from `record.__dict__`: alphabetically sorted, reserved attributes filtered, `_`-prefixed keys dropped (defensive), and values containing whitespace or `|` rendered via `repr` for grep safety. `tests/unit/test_logging_formatter.py` locks 22 unit tests covering rendering (all three modes), the ANSI palette, isatty auto-selection in both directions via fake streams, idempotence, reserved-attribute filtering, and the ANSI-vs-THEME alignment invariant. No production caller changed — Stage 2 wires the formatter into the CLI group callback, rewires `StructuredLogger`, and removes legacy `basicConfig` / duplicate `docker_logger` sites. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat(cli): --verbose/--quiet/--log-format root flags + StructuredLogger rewire (#279) Stage 2 of A3. Adds the root CLI flag surface for console verbosity and line-shape selection, rewires `StructuredLogger` through the root handler installed by `configure_root_logging` (from Stage 1), and removes two duplicate/legacy handler-install sites the plan called out. Root flags on `cli()`: - `-v/--verbose` → DEBUG on console - `-q/--quiet` → WARNING on console - `--log-format={pretty,plain,json}` (auto: pretty on TTY, plain otherwise) - `-v -q` together → `UsageError` Subcommand `--verbose` composition (symmetric option (a) — locked by the composition matrix in `tests/unit/test_logging_cli_flags.py`): `run/prepare/worker/convert --verbose` still bump the console to DEBUG when root did not receive `-q`. Root `-q` wins on console suppression; the orchestrator still receives `verbose=True` so per-trial `logs.yaml` records DEBUG. `StructuredLogger` now propagates to the root handler: - Removed the private `StreamHandler(sys.stdout)` and legacy `%(asctime)s - %(name)s - %(levelname)s - %(message)s` formatter. - Console output moves from `sys.stdout` to `sys.stderr` via the new root handler (aligned with A4's stdout carveout, #280). Downstream consumers piping tolokaforge's stdout for log lines should switch to `2>&1` or `--log-format=json` on stderr. - Records emit via `self.logger.log(level, msg, extra=...)`; extra keys that would collide with reserved `LogRecord` attributes are renamed with a `ctx_` prefix so `LogRecord.__init__` never raises. - `.logs[]`, `save_to_file`, `.strict`, and singleton behaviour are unchanged. Cleanups in this commit: - Removed the duplicate `docker_logger` handler installation in `Orchestrator.__init__` (records now propagate to root; the `setLevel` line stays to govern namespace threshold). - Removed `logging.basicConfig(level=logging.DEBUG, stream=sys.stderr)` in `adapter_commands.convert`; the root handler subsumes it. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * test(logging): canonical goldens + docs/CHANGELOG for structured logging (#279) Locks the exact bytes StructuredFormatter emits per mode with 10 golden files under tests/canonical/golden/logging/ (3 modes × {bare, scoped} + pretty {WARNING, ERROR, DEBUG, reserved_collision}) and a parametrised canonical test that compares byte-for-byte with a fixed clock, plus JSON-schema parseability for the two json__*.log goldens. docs/CLI.md gains a `## Structured logging` section documenting the root `-v / -q / --log-format` flags, the three format modes, and the precedence table with subcommand `--verbose`. docs/LOGGING.md replaces its § CLI Flags and § Console Output sections with current-state descriptions and adds § Log stream calling out stderr as the single stream for every tolokaforge log record. CHANGELOG adds two BREAKING entries under Unreleased (stderr move + line-shape change). .gitignore un-ignores tests/canonical/golden/**/*.log so the goldens survive the `*.log` catch-all. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(logging): cap root handler at requested level so -q silences docker INFO (#279) Review-response follow-ups: - configure_root_logging now calls handler.setLevel(level) so root -q is authoritative even against child loggers that raise their own effective level (e.g. Orchestrator.__init__ setting tolokaforge.docker to INFO). Without this cap, docker "Pulling image..." messages leaked through -q, contradicting docs/CLI.md and docs/LOGGING.md's precedence table. - Added test_root_quiet_silences_docker_namespace_even_when_child_level_is_info to lock the fix — the case that exposed the bug. - Added test_redaction_scrubs_secret_before_formatter_renders — composes install_global_redactor + StructuredFormatter to verify the record factory scrubs before the formatter renders (plan's redactor×formatter integration bullet, which slipped in Stage 3). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * test(logging): restore SecretManager singleton in redaction test (#279) The prior redaction test swapped the process-global _default_manager and log_filter cache but never restored them, breaking downstream tests (test_model_client::test_generate_nova_unwraps_input_key) that expect the CI-loaded SecretManager to have NOVA_API_KEY. Save/restore _default_manager, _cached_manager, and _cached_values in the same try/finally as the LogRecord factory. Matches the pattern in tests/unit/secrets/test_log_filter.py::_reset_singleton. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
* docs(plans): add A4 stdout-artifact stderr-progress plan (#280) Plan for milestone-11 A4: publish and enforce the CLI stdout/stderr contract. tolokaforge run and prepare emit the resolved absolute run-dir path on the LAST stdout line; everything else stays on stderr per A1+A3. Adds emit_artifact_path helper to _display.py, widens Orchestrator.run() return type from None to Path, fixes the silent- return-0 on zero-tasks path, canonical grep-guard forbids new print( in tolokaforge/cli. Depends on A1/A3. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat(cli): emit_artifact_path + Orchestrator.run returns Path + fail-loud on no-tasks (#280) Stage 1 of A4 (stdout is artifact, stderr is progress): - Add `emit_artifact_path(path)` to `tolokaforge/cli/_display.py` — the one sanctioned `sys.stdout` write in the CLI. Emits the resolved absolute path with a trailing newline, flushed. - Widen `Orchestrator.run()` return type from `None` to `Path`, returning the resolved absolute path of the timestamped run directory the orchestrator created. Callers ignoring the return value are unaffected. - Wire `tolokaforge run` to capture `output_dir = orchestrator.run()` and call `emit_artifact_path(output_dir)` as the last line of the successful path. Adjust the pre-run "Output base:" line to reflect that `evaluation.output_dir` is the base, not the timestamped run dir. - Convert the "No tasks found!" branch from silent `return` (exit 0) to `raise SystemExit(1)` so the zero-tasks failure surfaces per the stdout/stderr contract. Behaviour locked by new unit tests in `tests/unit/test_cli_display.py` (`TestEmitArtifactPath`) and `tests/unit/test_cli_stdout_contract.py` (`TestRunStdoutContract`). Docs, canonical grep-guard, and the `prepare` wiring land in Stages 2 & 3. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat(cli): prepare emits artifact path; lock stdout empty for read-only commands (#280) Wires ``emit_artifact_path(run_dir)`` at the tail of ``tolokaforge prepare``'s successful path, so the ``RUN_DIR=$(tolokaforge prepare --config … --run-dir X)`` idiom captures the same resolved absolute path A4's ``run`` command already publishes. Failure paths (bad config, orchestrator raise) exit non-zero with stdout empty — ``SystemExit`` short-circuits before the emit line runs. Extends ``tests/unit/test_cli_stdout_contract.py`` with: - ``TestPrepareStdoutContract`` — success emits one resolved-path line; bad-config and orchestrator-raise leave stdout empty. - ``TestReadOnlyCommandsStdoutIsEmpty`` — locks ``status`` / ``validate`` / ``config validate`` / ``assets stamp`` as never touching stdout, so a future ``print`` regression on any of them fails CI here. Stage 3 will add the canonical grep-guard forbidding any other ``print(`` / ``sys.stdout.write(`` in ``tolokaforge/cli/**/*.py``, plus the ``docs/CLI.md`` stdout/stderr contract section and the CHANGELOG entry. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * test(cli): canonical stdout grep-guard + docs/CLI + API + CHANGELOG for A4 (#280) Locks the stdout=artifact / stderr=progress contract that Stages 1 & 2 wired up: adds two canonical guards, publishes the contract in docs/CLI.md, updates the docs/API.md Orchestrator snippet, and records the surface change plus the "no tasks" exit-code flip in CHANGELOG. - tests/canonical/test_cli_display_invariants.py: - test_no_bare_stdout_write_in_cli — walks tolokaforge/cli/**/*.py (exempting _display.py + __init__.py) and forbids any `print(` or `sys.stdout.write(`. Regex anchored to start-of-line or whitespace so pprint / sprint / imprint / console.print do not match. - test_emit_artifact_path_is_exported — pins the sanctioned helper on _display's public surface. - docs/CLI.md: - Rewrites the "later stage will introduce" line — the stage has landed; the line now points to the new section. - Adds `## stdout / stderr contract` after `## Structured logging` with a per-command table, failure-mode text, the resolve() note, and the `RUN_DIR=$(tolokaforge run …)` idiom. - docs/API.md: Orchestrator snippet now shows `run_dir = orchestrator.run()` with a comment naming the return type. - CHANGELOG.md: three Unreleased entries — cli feat (new stdout surface), embedder note (Orchestrator.run return-type widening), breaking (zero-tasks exit code flip from 0 → 1). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * test(cli): lock emit flush + runtime empty-stdout on adapter/docker/analyze (#280) Review-response follow-ups: - test_emit_artifact_path_calls_flush now spies on sys.stdout.flush() directly. The prior capfd-based test would pass even without flush=True because line-buffered stdout flushes on newline before readouterr(). - Add runtime empty-stdout locks for adapter convert, docker status (no-SDK path), and analyze (missing trajectory). Grep-guard covers the print( regression path; these lock the non-print variants (click.echo, Console(stderr=False) slips) in commands the docs promise as empty-stdout. - Worker's runtime lock skipped — needs heavy queue-backed fixtures that don't exist cheaply; grep-guard remains the safety net. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
* docs(plans): add B2 --display toggle plan (#282) Plan for milestone-11 B2: DisplayMode enum {FULL,RICH,PLAIN,LOG,NONE}, --display root flag + TOLOKAFORGE_DISPLAY env var, selection-time Textual fallback for FULL, dual-mechanism silencing for --display=none (console.quiet + handler CRITICAL+1). Precedes B1. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat(cli): DisplayMode enum + select_display_mode + silence helpers (#282) B2 Stage 1 — pure/testable primitives, no CLI wiring yet: - DisplayMode enum (FULL/RICH/PLAIN/LOG/NONE) in _display.py. - select_display_mode() applies precedence explicit > TOLOKAFORGE_DISPLAY env > CI truthy > isatty > PLAIN. Invalid values raise ValueError. - silence_console() sets console.quiet=True on the shared console. - silence_root_logging() raises the tolokaforge sentinel handler above CRITICAL so no log record emits. - _textual_available() probes via importlib.util.find_spec. - 39 new unit tests locking every branch of the precedence, invalid inputs, idempotence, and foreign-handler isolation. Stage 2 wires --display flag on cli() and applies these helpers under --display=none; Stage 3 adds canonical guards + docs + CHANGELOG. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat(cli): --display root flag + TOLOKAFORGE_DISPLAY env var + silencing (#282) Adds the root ``--display={full,rich,plain,log,none}`` flag on ``tolokaforge/cli/main.py::cli()`` and wires the equivalent ``TOLOKAFORGE_DISPLAY`` env var. Resolves the effective mode via ``select_display_mode(explicit=..., env=os.environ)`` with the precedence chain locked in Stage 1 (explicit > env > CI > isatty), applies the Textual fallback (``full`` -> ``rich`` with a WARNING log line when ``import textual`` fails), and stashes the resolved ``DisplayMode`` on ``ctx.obj["display_mode"]`` for B1 / C3 consumers. On ``--display=none`` the group callback also calls ``silence_console`` and ``silence_root_logging`` so ``tolokaforge run --display=none`` produces empty stderr on success while ``emit_artifact_path`` still writes the run-dir path to stdout. Failure paths bypass the silencer: click's ``UsageError`` writes directly to stderr and uncaught exceptions surface via ``result.exception`` (in production, ``sys.excepthook`` prints the traceback). Behaviour locked in ``tests/unit/test_cli_display_flag.py`` (21 tests): five explicit modes propagated to ctx.obj, invalid flag/env values exit 2, env-var / CI precedence rows, Textual fallback with and without a fake ``textual`` module installed, ``--display=none`` silences stderr on success under ``-v`` / ``-q`` / ``--log-format=json`` while the artifact path still lands on stdout, and orthogonality against ``--log-format``. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * test(cli): canonical DisplayMode surface + docs/CLI + CHANGELOG for B2 (#282) Stage 3 of the B2 display-mode toggle. Locks the flag / env-var operator surface at the canonical tier and publishes the user-facing documentation. - Extend `tests/canonical/test_cli_display_invariants.py` with four tests: `test_display_mode_enum_surface` pins the FULL/RICH/PLAIN/LOG/NONE member order and CLI-literal values; `test_select_display_mode_is_exported` and `test_silence_root_logging_is_exported` pin the selector and silencing helpers on their modules; `test_display_env_var_literal_is_TOLOKAFORGE_DISPLAY` locks the operator env-var literal. - Add `docs/CLI.md § Display modes` between `§ Structured logging` and `§ stdout / stderr contract` covering the mode table, precedence, `--log-format` composition, `--display=none` semantics, failure-path bypasses, invalid-env-var behaviour, and `ctx.obj["display_mode"]`. Append a cross-reference sentence to `§ stdout / stderr contract`. - Refresh `§ Why a shared surface` to describe the `--display` toggle as current state, not as future work. - Add an Unreleased/Feat CHANGELOG entry for the flag, env var, and `--display=none` silence semantics. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * docs(cli): trim future-work refs in Display modes section (#282) Reviewer nit: docs/CLI.md described B1/C3 as future work in two places. Docs describe current state per AGENTS.md Core Rule 8 — CHANGELOG and plan file already reference the tickets. Rewrites both sentences to describe the current selection semantics without forward-tense. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
#340) * docs(plans): add B1 Rich Live progress panel plan (#285) Plan for milestone-11 B1: LiveRunDisplay panel with left trial list, right focused-trial summary, bottom cost/tokens/eta bar. RunDisplayEvents Protocol threaded through OrchestratorDeps → ConductorContext → TrialRunner → _AgentMetricsSink. Right pane derived from structured event fields (turn_count, last_event_kind), NO free-form log lines. threading.Lock on every event handler for atomicity under N workers. Focus follows lifecycle transitions only, NOT trial_progress ticks. Depends on A1, A3, B2. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat(cli): LiveRunDisplay + RunDisplayEvents Protocol (#285) Stage 1 of B1 (milestone-11 Terminal DX). Adds tolokaforge/cli/_run_display.py with the panel class and the seven-method RunDisplayEvents Protocol the orchestrator/conductor/runner will emit into (wiring lands in Stage 2). LiveRunDisplay implements the Protocol directly (single-object, no wrapper), gates activation via for_mode(DisplayMode) with a _NoopDisplayCtx fallback for PLAIN/LOG/NONE, re-points every sentinel-tagged root handler's stream to Rich's live-aware sys.stderr on enter, and restores on exit. threading.Lock guards every event handler body so per-turn counter mutations under 12 concurrent workers are atomic. Focus follows lifecycle transitions only — trial_progress mutates counters and turn_count but never bumps last_update_ts. Unit coverage in tests/unit/test_run_display.py (43 tests): protocol shape, for_mode gate over five modes, event-to-card transitions, kwarg-only enforcement, D7 focus-follow rules under interleaved progress + just-completed-while-others-run, D8 concurrency (12 threads x 1000 progress events + started/completed variant, exact-equal counters), _visible_cards window trimming, _format_bottom_bar over the D6 case matrix (cost/tokens/eta edge cases + verbatim locked example), sentinel-handler stream re-point + restore. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat(cli): wire LiveRunDisplay + RunDisplayEvents through Orchestrator (#285) Stage 2 of B1. Thread the `RunDisplayEvents` Protocol through `OrchestratorDeps` → `ConductorContext` → `InProcessConductor` → `TrialRunner` → `_AgentMetricsSink`. Emit the seven lifecycle events from their natural sites: `run_started` before the thread pool, `trial_started` in `submit_one`, `trial_completed` / `trial_failed` in the wait loop, `judgment_scored` after grading, `trial_progress` per LLM generation, `run_finished` before returning the run directory. `tolokaforge run` wraps `orchestrator.load_tasks()` + `orchestrator.run()` in `with LiveRunDisplay.for_mode(mode):`, where `mode` is `ctx.obj["display_mode"]`. `emit_artifact_path` fires after the `with` block exits so Live redraw artefacts cannot contaminate the single stdout line. New defaults: every added field / kwarg defaults to `_NULL_EVENTS` so existing callers work unchanged. Locked by two new unit test files (27 tests) plus the existing 43 Stage 1 tests. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * test(cli): canonical SVG goldens + docs/CLI + CHANGELOG for B1 (#285) Lock the rendered shape of the LiveRunDisplay Rich Live panel at 80-column and 120-column widths via byte-for-byte SVG goldens under tests/canonical/golden/run_display/. Test replays a fifteen-event sequence into a fresh display under a monotonic frozen clock, renders display._build_layout() into a recording Console, and compares the export_svg output byte-for-byte. Regenerate on intentional layout changes with `pytest --update-canon`. docs/CLI.md § Display modes now describes the panel — three regions (trial list / focused-trial summary / bottom status bar), the D6 bar format, the RunDisplayEvents Protocol, and the log-line coexistence posture. CHANGELOG entry surfaces the new panel + event Protocol for downstream consumers. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(cli): LiveRunDisplay re-renders on every event via Live.update (#285) Round-3 review-response fix. Stage 3 report flagged that the panel would freeze at "empty startup" in production because Rich's Layout binds child renderables at construction and never re-renders on state change. Two fixes: 1. Add _refresh_live_locked() called at the end of every event handler (still under the lock) that rebuilds the layout via _build_layout() and pushes it to self._live.update(). Under auto_refresh=True at 4Hz, the panel now reflects current state. 2. Switch self._lock from threading.Lock to threading.RLock. The event-handler bodies hold the lock, then _refresh_live_locked calls _build_layout, whose render helpers (_visible_cards, _render_right_pane, _render_bottom_bar) re-acquire the same lock. A non-reentrant Lock deadlocks; RLock allows the same thread to acquire multiple times. New test test_events_push_updated_layout_to_live installs a stub Live and asserts each event handler triggers exactly one update() call — locks the "panel does not freeze" invariant. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * refactor(core): move RunDisplayEvents Protocol to core.run_display_events (#285) Reviewer's major finding: placing the Protocol in cli/_run_display.py made tolokaforge/core/{orchestrator,conductor,runner}.py import from tolokaforge.cli — a layer inversion. Importing orchestrator.py would transitively load Rich even in runtime consumers (worker container, gRPC runner, cloud-runtime trial-plane) that never render. Fix: extract the pure interface (RunDisplayEvents, _NullRunDisplayEvents, _NULL_EVENTS) to tolokaforge/core/run_display_events.py. Core modules import from there. cli/_run_display.py imports the same Protocol and implements the concrete Rich-bound LiveRunDisplay. Also folds two reviewer nits: - _visible_cards now holds the lock across the whole sort/partition. - Dropped "(D7)" plan-section citation from the trial_progress comment. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
* docs(plans): add A2 --version + grouped --help plan (#278) Small plan (2 stages): _GroupedCommandsGroup subclass rendering Runs/Tasks/Docker/Config/Assets/Adapters headings + @click.version_option on root; Stage 2 docs. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat(cli): --version + grouped --help sections (#278) Adds `_GroupedCommandsGroup(click.Group)` in `tolokaforge/cli/main.py` that renders the root `Commands:` block as six fixed-order sections — Runs, Tasks, Docker, Config, Assets, Adapters — with alphabetical ordering within each section. Wires the root group as `cls=_GroupedCommandsGroup` and adds `@click.version_option(package_name="tolokaforge")` so `tolokaforge --version` prints `tolokaforge, version <version>` sourced from `importlib.metadata`. Drift protection: a registered top-level command missing from `_GroupedCommandsGroup.COMMAND_GROUPS` raises `RuntimeError` at `--help` time. Per-command `--help` (`run --help`, `docker --help`, `docker up --help`, …) is unchanged — grouped formatting is confined to the root group. Behaviour locked in `tests/unit/test_cli_grouped_help.py` (13 tests): version matches installed metadata + writes to stdout, six headings in fixed order, commands under correct sections, alphabetical within Runs, registered-but-unmapped command raises, subcommand help retains Click's default layout. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * docs(cli): Root help layout + --version in docs/CLI.md + CHANGELOG (#278) Adds a `## Root help layout` section to `docs/CLI.md` describing the six fixed-order section headings (Runs / Tasks / Docker / Config / Assets / Adapters), the alphabetical-within-section rule, the `tolokaforge --version` output, and the drift-protection contract enforced by `_GroupedCommandsGroup.COMMAND_GROUPS`. Adds the matching CHANGELOG Unreleased Feat entry. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
* docs(plans): add A5 dashboard-URL banner plan (#281) Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat(cli): run-start/end banner helpers + resolve_run_directory + format_duration extraction (#281) Add `_run_banner` module with `print_run_start_banner` / `print_run_end_banner`. Both helpers accept the shared `console`, wrap the resolved absolute `file://` URL in Rich `[link=URL]…[/link]` markup (styled via the `link` theme token), and emit the `→ Run:` / `→ Report:` / `✓ Run complete in <duration>` / `✗ Run failed in <duration>` / `→ Browse: tolokaforge browse <run-id>` framing on stderr. Extract `format_duration(seconds)` into `_display.py` — shared `MM:SS` under one hour / `HH:MM:SS` above — and route `_run_display._format_eta` through it so B1's bottom-bar shape stays byte-identical. Add module-level `resolve_run_directory(base_output_dir)` in `orchestrator.py` that computes the `(run_id, output_dir)` pair callers (CLI banner, integration tests) need before the orchestrator starts. `Orchestrator.run()` keeps its internal computation for now; the wiring to consume the pre-resolved pair lands in the next commit. Behaviour locked: * `tests/unit/test_run_banner.py` — duration boundaries incl. fractional truncation and hour crossover; banner writes exactly two / three lines with the documented glyphs and prefixes; `file://` URL is absolute and OSC-8-wrapped; success/failure variants render the correct outcome line + browse invocation. * `tests/unit/test_orchestrator_resolve_run_directory.py` — empty basename raises `ValueError` naming `evaluation.output_dir`; successive calls advance with the clock; a frozen clock yields a fully-determined pair. * `tests/canonical/test_run_banner_goldens.py` + three SVGs under `tests/canonical/golden/run_banner/` — start banner, end-success (MM:SS branch), end-failure (HH:MM:SS branch) pinned byte-for-byte via `Console.export_svg(unique_id="tolokaforge-run-banner")`. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat(cli): wire A5 banners into tolokaforge run + Orchestrator.run(run_id, output_dir) kwargs (#281) Bookend every ``tolokaforge run`` invocation on stderr with the A5 start / end banners (``→ Run:`` + ``→ Report: file:///…`` up top; ``✓ Run complete in <duration>`` / ``✗ Run failed in <duration>`` + ``→ Report:`` + ``→ Browse: tolokaforge browse <run-id>`` at the bottom). The end banner fires on both success and failure via a ``try:``/``finally:`` — on failure it renders before the exception continues propagating to Click. Thread the pre-resolved ``(run_id, output_dir)`` pair through ``Orchestrator.run(*, run_id=None, output_dir=None)`` so the CLI can name the run directory in the start banner before the run begins. When both kwargs are omitted (existing test / notebook callers), ``run()`` calls ``resolve_run_directory`` itself; supplying exactly one raises ``ValueError``. Delete the two superseded ``console.print`` lines (``✓ Run complete!`` and ``Results saved to:``) — the end banner now conveys both signals. ``emit_artifact_path`` continues to fire on the success path only, after the end banner. CLI integration coverage in ``test_run_banner_cli_integration.py`` locks the stderr landmarks, the ordering (``__enter__`` → run → ``__exit__`` → end banner → stdout), the failure-path ordering (no ``emit_artifact_path`` on raise), the ``--display=none`` silence contract, and the pre-resolved pair reaching the orchestrator. Existing stub orchestrators widened to ``def run(self, **_: object)`` to swallow the new kwargs. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * docs(cli): Run banner section in docs/CLI.md + CHANGELOG (#281) Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
…rlay (#283) (#357) * docs(plans): add B3 budgets + fallback models plan (#283) Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat(cli): parse_duration + pricing.reload_pricing overlay (#283) Stage 1 of B3. Two pure modules with unit tests; no CLI wiring yet. - New tolokaforge/core/duration.py exports parse_duration(spec) -> float seconds. Accepts compound tokens (1h30m, 1h30m45s, 1d12h) and fractional units (1.5h); rejects empty, bare number, unknown unit, and negative input with a ValueError naming the offending input. - tolokaforge/core/pricing.reload_pricing gains an overlay_path kwarg. JSON/YAML suffix-detected; field-level merge onto the shipped baseline; missing file raises FileNotFoundError; malformed or wrong-shape overlay raises ValueError. - tests/unit/test_parse_duration.py — 20 cases (valid + invalid). - tests/unit/test_pricing_overlay.py — 11 cases (regression + YAML/JSON override + additive model + field-level merge + error branches). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat(core): BudgetTracker + LIMIT_HIT.json marker + Orchestrator integration (#283) Ship the graceful-shutdown machinery B3 needs on top of the existing cost-cap path. ``tolokaforge/core/budgets`` introduces the ``BudgetTracker`` Protocol with three concrete trackers (``CostBudget`` / ``TimeBudget`` / ``SampleBudget``), a first-hit-wins ``CompositeBudget``, and a ``LimitHitMarker`` Pydantic model on-disk shape written under ``<output_dir>/LIMIT_HIT.json``. ``Orchestrator.OrchestratorDeps`` grows one field (``budget: CompositeBudget | None``); ``Orchestrator.run()`` records generation cost + trial-termination on every terminal branch, polls the composite at the schedule gate, writes the marker on the first hit and sets ``_stopped_reason`` so the CLI can shape the run-end banner in Stage 3. Legacy ``compute.max_budget_usd`` continues to work — the orchestrator promotes it to a single-tracker ``CompositeBudget`` seeded with prior spend (from ``_collect_existing_cost``) when no ``deps.budget`` is supplied. Case F pins this observable-shape parity. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat(cli): --cost-limit/--time-limit/--sample-limit/--fallback-models/--model-cost-config + FallbackLLMClient + B1 amber/red meter (#283) Stage 3 of the B3 budgets plan. Five new `tolokaforge run` flags land end-to-end; the B1 cost meter picks up budget context; the A5 end banner gains a `stopped_reason` variant; and a `FallbackLLMClient` lets a batch survive provider outages via an ordered per-trial cursor. * `--cost-limit <usd>` / `--time-limit <duration>` / `--sample-limit <n>` compose into a `CompositeBudget` via `make_budget` and drive the graceful-shutdown path landed in Stage 2. * `--fallback-models <m1,m2,...>` builds an `agent_client_factory` wiring `FallbackLLMClient(primary, [m1,...])` — cursor advances on any exception surfacing out of `LLMClient.generate` after the inner retry. * `--model-cost-config <path>` calls `pricing.reload_pricing(overlay_path=...)` before orchestrator construction so every cost-attribution site sees the merged rates. * `LiveRunDisplay(cost_budget_usd=...)` renders the bottom-bar cost segment in the `warn` theme token at ≥80% of budget, `error` at ≥100%; unset budget renders unchanged (existing goldens byte-identical, two new goldens land for the amber/red states). * `print_run_end_banner(stopped_reason=...)` renders `⏸ Run stopped (<reason>) in <dur>` when a `LIMIT_HIT.json` marker is present under the run directory. `stopped_reason` defaults to `None`, preserving A5's signature. * `OrchestratorDeps.agent_client_factory` is additive with `None` default; both `run()` and `run_worker()` route the primary agent client through the factory when set. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * docs(cli): B3 budget/fallback/pricing sections in docs/CLI.md + OUTPUT_FORMAT + CHANGELOG (#283) Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
…361) * docs(plans): add B4 run --dry-run plan (#284) Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat(core): extract system_prompt module + dry_run materializer (#284) Stage 1 of B4 (--dry-run): - tolokaforge/core/system_prompt.py — pure ``build_system_prompt`` helper. Priority chain (inline → __adapter__ → file path → legacy main_policy.md split → minimal default) is unchanged; only local file I/O. - tolokaforge/core/conductor.py — ``InProcessConductor._build_system_prompt`` now delegates to the module-level helper. Behaviour preserved. - tolokaforge/core/dry_run.py — ``DryRunSample`` value type, ``materialize_dry_run_sample`` (first-turn payload without Docker or HTTP), and ``load_tasks_for_dry_run`` (adapter + tasks without the TypeSense preflight). - tests/unit/test_system_prompt.py — locks the five branches + conductor-delegation parity. - tests/unit/test_dry_run.py — locks the sample fields, the literal / placeholder user-prompt split, the no-HTTP guarantee (httpx.Client.send + litellm.completion patched to raise), the TypeSense-preflight skip, and byte-for-byte tool-spec parity with ``TrialArtifactWriter.write_tools_schemas``. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat(cli): --dry-run flag + panel renderer + integration tests (#284) Wires the Stage 1 dry-run materializer into ``tolokaforge run`` via ``--dry-run`` / ``--dry-run-samples`` flags. Renders one Rich panel per task on stderr (system prompt, first user message, sanitized tool spec, resolved agent / judge / runtime), then exits 0 without creating a run directory, entering ``LiveRunDisplay``, or issuing any HTTP call to a provider. ``--display=none`` silences the panels via the shared console. ``--dry-run --resume`` and ``--dry-run-samples`` without ``--dry-run`` fail loudly with ``click.UsageError``. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * test(cli): canonical dry-run SVG goldens + docs/CLI + CHANGELOG for B4 (#284) Locks the rendered ``tolokaforge run --dry-run`` panel byte-for-byte at 80 and 120 columns via ``Console.export_svg`` under a deterministic :class:`DryRunSample` fixture. Expands ``docs/CLI.md § Dry run`` with the full contract (what the flag does, panel body order, sample selection, zero-HTTP guarantee, composition with other flags, stream posture, TypeSense side-effect note) and adds a CHANGELOG entry under Unreleased Feat. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
…ly pending trials (#286) (#364) * docs(plans): add B5 --resume plan (#286) Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat(core): resume resolver + ResumePlan (#286) Adds `resolve_resume_run_directory` and `RunStateManager.describe_resume_plan()` + `ResumePlan` dataclass in `tolokaforge/core/resume.py`. The resolver returns `(run_id, run_dir)` for an existing run directory (reading the canonical run_id from `engine_run_state.json` or falling back to `run_state.json` / `run_dir.name`); it raises when neither metadata file is present. `describe_resume_plan` returns counts of `completed`, `already_done` (behavioural-failed and passed trials that will be skipped) and `to_retry` (pending, running, infra-failed trials that will re-execute), using the existing `is_completed` predicate. Pure helpers — the CLI wiring lands in Stage 2. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat(cli): --resume + --run-dir wiring + Resume start banner variant (#286) `tolokaforge run --resume --run-dir <existing>` now reuses the passed directory verbatim: the resolver reads engine_run_state.json, the state manager's describe_resume_plan drives the summary and idempotent no-op, and the start banner shows `→ Resume: <run-id>` instead of `→ Run:`. Flag guardrails: `--resume` without `--run-dir` and `--run-dir` without `--resume` both raise UsageError. On a fully-completed run the CLI prints the "Nothing to do" line, emits the artifact path, and exits 0 without constructing an Orchestrator or touching Docker. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat(worker) + docs: resume summary log + docs/CLI + RUNNER + REFERENCE + CHANGELOG for B5 (#286) - Worker INFO log line on reattach to a run dir whose queue holds completions: "Reattaching to run dir <run-id>: X/N completed, K pending in queue." Silent on cold-start queues. - docs/CLI.md § Resume documents --resume + --run-dir, idempotent no-op, Resume start-banner variant, budget composition per-invocation, and cross-links the queue-worker resume section. - docs/RUNNER.md § Resuming a queue-worker run documents that worker restart is inherent resume via the durable queue, with the reattach INFO log line and --reset-queue behaviour. - docs/REFERENCE.md § Orchestrator rewritten to reflect the real Orchestrator(config, resume=, deps=, project=) constructor and orchestrator.run(run_id=, output_dir=) / prepare_run / run_worker entry points. - CHANGELOG unreleased Feat entry names the fix + the new behaviour. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
…397) * feat(cli): fail-fast auth + panel error rendering + phase visibility + PATH docs Post-B5 smoke revealed four papercuts. This commit ships the core-level fixes and the terminal-panel improvements; a follow-up plan (.claude/plans/clever-nibbling-haven.md) covers the tolokaforge.cli → tolokaforge.dx namespace refactor + REPL. Core (library-visible): - llm/client.py::_should_retry_exception short-circuits on openai.AuthenticationError. A bad key fails on attempt 1 (~1s) instead of exhausting 5 retries with exponential backoff (~35s per trial). - orchestrator.py::_is_auth_failure classifies auth-shaped API_ERROR trajectories as non-retryable. - run_display_events.py: new phase_changed(*, phase, detail) protocol method + null-object no-op. Fires at Docker startup milestones so consumers can render "Starting services..." during the pre-run_started window that used to show 0/0. Terminal panel (cli/_run_display.py): - Auth-failure top banner on first auth-shaped trial_failed, with provider-specific remediation hints. Existing goldens byte-identical when banner is absent. - Error rendering in left pane (truncated tail after the row) + right pane (full FAILED heading + first-line error + hint) for failed trials. - Startup phase visibility: bottom bar renders "Starting services… (docker compose up)" while total_trials==0 and a phase event has fired. Transitions cleanly to counters formatter when run_started fires. - Text.from_markup switch is guarded (has_markup bool) so non-error panels stay byte-identical to pre-change goldens. Docs (Stage 2): - README.md, AGENTS.md: replace `uv run tolokaforge` with bare `tolokaforge`. Added `uv tool install --editable . --python 3.12` install line so the console script is globally on PATH. Test updates: - _RecordingEvents / _FakeEvents get phase_changed(**_) methods. - Ordering tests filter phase_changed out before asserting on first/last lifecycle events. - Protocol size test 7 → 8 methods. - SVG goldens regenerated for the new failed-trial rendering (byte diff limited to the failed row + its error line). 3190 unit + canonical tests green. * refactor(dx): namespace tolokaforge.cli → tolokaforge.dx + ADR-0019 + extras split BREAKING: tolokaforge.cli.* modules renamed to tolokaforge.dx.* (and tolokaforge/dx/cli/* for the Click command modules). Library-only imports (from tolokaforge.core.orchestrator import Orchestrator) are unaffected. Records the pluggable front-end pattern per ADR-0011: - RunDisplayEvents (already engine-side in tolokaforge.core) is the front-end seam. - tolokaforge.dx is the reference terminal front-end. - Future front-ends (web dashboard, alternative TUI) plug in via the same Protocol under tolokaforge.dx.<medium> or as a separate distribution. pyproject.toml: - rich moved from [project.dependencies] to [project.optional-dependencies].dx. - New [project.scripts] tolokaforge = "tolokaforge._entry:main" with a stdlib-only fallback shim that prints an install hint when [dx] extras are absent. Docs: README, docs/CLI.md, docs/LOGGING.md, docs/PERFORMANCE.md, docs/architecture/README.md, AGENTS.md, ADR-0011 (Accepted + follow-through row), ADR-0019 (new), CHANGELOG Unreleased breaking entry. 3190 unit+canonical tests still green. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat(dx): interactive REPL via click-repl (Stage 6) Adds `tolokaforge` (no args) / `tolokaforge repl` interactive shell. Free-form Click passthrough: tab-completion of subcommands, flag names, file paths; command history in ~/.tolokaforge_history; exit via `exit` / `quit` / Ctrl-D. Session-scoped: root flags (`-v/-q/--display/--log-format`) supplied at REPL entry apply to every command until the REPL exits. Dependencies added to [dx] extras: click-repl>=0.3.0, prompt-toolkit>=3.0.51. Library-only installs (pip install tolokaforge) are unaffected; running `tolokaforge` without the [dx] extras prints the install-hint from the fallback shim. Tests: 5 new tests in tests/unit/dx/test_repl.py — REPL registration in COMMAND_GROUPS, --help rendering, wiring invocation (mocking click_repl.repl so tests don't need a TTY). Total suite: 3195 green. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(deps): cap click<8.2 for click-repl 0.3.0 compatibility Runtime smoke of the Stage 6 REPL blew up: AttributeError: property 'protected_args' of 'Context' object has no setter click 8.2 removed the setter for ``Context.protected_args``, but the latest click-repl (0.3.0, no newer version on PyPI) still assigns to it in its REPL entry (click_repl/_repl.py:134). The test suite passed because Stage 6 mocks ``_click_repl`` to keep tests headless — the real interaction only broke when uv resolved click 8.2.1 in the isolated ``uv tool`` env. Pin ``click>=8.1.0,<8.2`` in [project.dependencies] with a NB comment naming the trigger and the lift condition. Verified end-to-end: $ printf 'run --help\nexit\n' | tolokaforge tolokaforge interactive shell. Type `help` for commands, `exit` to quit. Usage: tolokaforge run [OPTIONS] ... 3195 unit + canonical tests still green. * feat(dx): Rich Live panel — log discipline, services widget, infra sub-panel, global trial index Refit the Rich Live run panel around the failure modes surfaced by a real smoke run: - `_LogSink` (500-record ring) replaces the sentinel root handler for the Live lifetime. INFO/DEBUG records land in the buffer; WARNING+ still writes to the pre-Rich stderr so real problems remain visible above the panel. Fixes the "panel scrolls off-screen during Docker boot" regression. - `phase_changed(services=…)` — the orchestrator now carries a `ServiceSnapshot` list on `starting_services` / `services_ready` so the panel renders a compact services widget during the startup window. Widget disappears once trials dispatch. - `trial_provisioned(trial_id, containers, endpoints)` — new `RunDisplayEvents` method fired by `ProvisioningTrialExecutor` after `runtime_backend.await_ready` returns. `RuntimeBackend` gains `get_infrastructure_snapshot(handle) -> list[ContainerSnapshot]` (per-trial docker `ps` for `PerTrialRuntimeBackend`, shared-stack snapshot for env-manifest `SharedStackRuntimeBackend`, synthesised single-container fixture for `InMemoryRuntimeBackend`). Focused-trial right pane now renders an Infrastructure sub-panel when the trial has containers. - `trial_started(total_index=…)` — the orchestrator hands out a run-wide 0-indexed counter alongside `trial_index`. Left-pane rows become `⏳ [17/500] task_c · 0`, padded to the width of the total count so rows stay aligned. - Adaptive `_build_layout` sizes the trials/focused row to `min(len(visible_cards) + 2, viewport - reserved - 1)`. Three trials in a tall terminal no longer waste 35 rows. Ninth Protocol method plus new TypedDicts (`ServiceSnapshot`, `ContainerSnapshot`) are additive on the engine seam per ADR-0011 Pattern A; `_NullRunDisplayEvents` and every concrete backend gain the no-op / real overloads in the same commit. Golden SVGs regenerated (`panel_80.svg`, `panel_120.svg`, and the two budget goldens) to capture the `[N/M]` prefix and adaptive main region. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat(dx): Textual TUI for tolokaforge run --display=full New module tolokaforge/dx/tui.py exposes TextualRunApp, a Textual App that consumes the same RunDisplayEvents seam the Rich Live panel does. Layout: Header + one-line status bar + trial-list (scrollable, keyboard- navigable) + focused-trial pane + TabbedContent with Overview / Logs / Services / Infra / Errors + Footer legend. Bindings: j/k or arrows for selection, PgUp/PgDn for ~20-row jumps, Home/End, 1-5 for tabs, l for Logs, ? for modal help, q to exit the UI (Ctrl-C still kills the run). Event dispatch is thread-safe: RunDisplayEvents methods are called from worker threads and route to the app's asyncio loop via App.call_from_thread; events fired before on_mount buffer into a pending queue and drain deterministically once widgets are ready. Handlers never raise — a bad payload is logged at WARNING and dropped. Log records feed through the same _LogSink the Rich panel installs; Logs and Errors tabs read from the shared ring buffer. LiveRunDisplay.for_mode(DisplayMode.FULL) returns TextualRunApp when textual is importable and falls back to the Rich panel with a WARNING log line otherwise. Dep: textual>=0.85.0 added to [project.optional-dependencies].dx. Docs: new docs/CLI.md § "Full TUI (--display=full)" with keybinding table; ADR-0019 refreshed to reference "Rich + Textual" and both production impls; CHANGELOG Unreleased Feat entry. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(dx): stop Rich Live from stacking + add boot spinner + fix empty services widget Three defects from the Stage 1 live-panel smoke: 1. Panel copies stacked instead of redrawing in place. Rich Live in inline (screen=False) mode redraws only when the total rendered height is stable between refreshes; the adaptive layout let banner and services regions grow/shrink the total, so Live re-anchored and printed fresh copies. Fix: pin total height to `viewport - 1` and have optional regions steal rows from `main` instead of lengthening the renderable. 2. Services widget rendered header row + zero data rows. A `Table(expand=True)` inside a `Panel` inside a `Layout(size=len(services)+3)` clipped its data rows when Rich's expand/ratio math clashed with the tight size cap. Fix: drop the Table; render one Text line per service (glyph + name + status + ports). Always visible, no sizing games. 3. No live indicator during the 10-30s Docker-boot / runtime-connect window. Fix: `rich.spinner.Spinner("dots")` in the bottom bar during the phase window — animates at Live's 4 fps tick so the user sees work in progress. Golden SVGs regenerate for the new stable-height layout (main now fills the viewport rather than being sized to visible-card count). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(dx): stop Live from stacking + parallelise coding example Two smoke-derived fixes: 1. Rich Live stacking (persisted after the earlier stable-height fix). Root cause: five `console.print(...)` calls inside `with LiveRunDisplay:` in `run` (Loading tasks…, Found N tasks, Running M trials per task…, Resuming…). Rich Live tries to move the panel down and print above it, but combined with the panel's own phase transitions the cursor math loses the anchor and re-appends. The panel already conveys this info through `phase_changed` / `run_started`, so the console prints were redundant. Deleted; error path routes through `logger.error` which the `_LogSink` forwards to the wrapped stream above the panel. 2. `examples/native/coding/run_config.yaml` had `workers: 1`, forcing sequential trials in a 2-task example (default is 8). Bumped to 2 so the example actually parallelises and the panel has something to show. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(dx): route WARNING+ log records through Live-owned console to stop stacking Third-pass stacking fix. Prior fixes (stable-height layout, console.print removal) closed the obvious channels but WARNING+ log records were still writing directly to the raw sys.stderr stream — the one captured by the sentinel handler *before* Rich Live installs its stderr redirect. Direct writes bypass Live's cursor bookkeeping: Live decrements by its last render height, but the terminal has already scrolled, so the next refresh anchors below the prior panel and duplicates stack. Fix: `_LogSink` now takes a `print_above: Callable[[str], None]` callback instead of a raw stream. `LiveRunDisplay.__enter__` binds it to `self._live.console.print(...)` — Rich Live intercepts prints on its own console, lifts the cursor above the live region, prints, then re-renders. Panel stays anchored, warnings still visible above it. `TextualRunApp` passes a no-op — Textual owns the terminal screen and reads WARNING records from the ring buffer via the Logs / Errors tabs, so no external write is ever needed. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * docs(plans): add plan for issue #392 — kill panel stacking Diagnostic-first plan: Stage 1 lands _StderrProbe (env-gated), Stage 2 generalises LiveRunDisplay.__enter__ to sweep non-root loggers that would bypass Rich Live's cursor coordination. Grounded via run_python repro that pinned litellm's child StreamHandlers as the remaining leak. plan-critic approved after one revision round. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat(dx): add env-gated _StderrProbe diagnostic for LiveRunDisplay Introduce _StderrProbe as a private diagnostic tap on the underlying sys.stderr stream object. When TOLOKAFORGE_STDERR_PROBE points at a file path, LiveRunDisplay.__enter__ wraps the stream's write method (before Rich Live installs its redirect proxy) and records every raw write with an ISO timestamp, caller file:line, chunk repr, and top-5 stack frames — then delegates to the original write so terminal output is unaffected. On __exit__ the write is restored and the log closed. Wrapping the stream object (not the sys.stderr name) is load-bearing: libraries such as litellm install StreamHandler(sys.stderr) at import time, capturing the stream object; rebinding the name post-proxy would miss the writes this diagnostic exists to catch. This is Stage 1 of the plan for #392; Stage 2 uses the probe's output to identify and close the litellm child-logger stderr-bypass channel. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(dx): neutralise child-logger stderr-bypass handlers for Live lifetime `LiveRunDisplay.__enter__` now sweeps every non-root logger in `logging.root.manager.loggerDict` (skipping `PlaceHolder` entries, which have no `.handlers` attribute) and removes any `StreamHandler` whose `.stream` is one of the pre-Live-captured terminal streams (`sys.stderr`, `sys.stdout`, `sys.__stderr__`, `sys.__stdout__`). For loggers with `propagate=False`, a fresh `_LogSink` is additionally installed so the records still surface through the panel instead of being silently dropped (AGENTS.md Core Rule 1). `__exit__` restores every removed handler and drops any installed child-logger `_LogSink`, symmetric with the existing root-handler restore. This closes the litellm `LiteLLM` / `LiteLLM Router` / `LiteLLM Proxy` stderr-bypass channel that caused the Rich Live panel to stack duplicate copies during trial execution. The predicate is stream-identity based — no hardcoded logger names — so it is general and does not couple `dx` to litellm internals. Acceptance criterion (per plan revision): "zero **bypassing** writes" during trial execution — no write to the captured stderr whose stack lacks a Rich render or `_LogSink.print_above` frame. Verified live via `TOLOKAFORGE_STDERR_PROBE=/tmp/probe-392-stage2.log scripts/with_env.sh uv run tolokaforge --display=rich run --config examples/native/tool_use/run_config.yaml` — probe log contains only Rich-coordinated writes; no litellm frames; panel rendered as a single anchored copy for the full 82-second run. Closes #392. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * chore(dx): drop issue-attribution + rulebook citations from source comments Per code-review skill §7a: delete issue #392 attributions from a section header and a test docstring, delete an AGENTS.md rule citation from a test docstring, and delete a SecretManager grep-guard citation from the _STDERR_PROBE_ENV_VAR docstring. Mechanical review-fix; no behaviour change. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * chore(deps): mirror textual into dev group so tui tests collect in CI The Stage 2 Textual TUI commit added `textual>=0.85.0` to `[project.optional-dependencies].dx` and `tests/unit/dx/test_tui_app.py` but did not extend the dev-group mirror alongside `rich`, `click-repl`, and `prompt-toolkit`. The gap is pre-existing on `feat/terminal-dx`; PR #397 is the first push from a branch based off that head to surface it in CI, where `test-smoke` fails at collection with `ModuleNotFoundError: No module named 'textual'`. Adding `textual>=0.85.0` to `[dependency-groups].dev` completes the mirror so `uv sync --dev` installs the terminal front-end deps that `tests/unit/dx/` exercises. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
* docs(plans): add plan for issue #394 — boot-window log tail widget 3-stage plan: filter+render helpers, wire optional region into _build_layout (with clamp preserving the stable-height invariant from #392), new canonical SVG goldens for the boot state. Pure consumer of the existing _log_buffer — zero engine changes. plan-critic approved after two revision rounds (TZ-stable UTC timestamps + clamped max_lines derived from granted region height to keep 'most-recent-last' correct on small terminals). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat(dx): add boot-log render helpers + filter predicate (stage 1 of #394) Introduces two pure module-level helpers in ``tolokaforge/dx/live_panel`` that Stage 2 will wire into the Rich panel's boot window: - ``_docker_boot_records`` — trailing-dot prefix filter keeps only ``tolokaforge.docker.*`` records, excluding the bare namespace logger and unrelated siblings. - ``_render_boot_log_tail`` — renders the last ``max_lines`` filtered records as ``HH:MM:SS.mmm | short-name | message`` inside a ``Panel(title="Boot log")``. Timestamps render in UTC so the byte output is TZ-stable across dev-box and CI; ``int(msecs)`` truncation caps the millisecond column at 3 digits. Unit tests exercise real ``logging.LogRecord`` instances (via ``logging.makeLogRecord``) — no mocks — covering the trailing-dot boundary, the last-``max_lines`` tail contract, and the ``msecs = 999.6`` truncation edge case. Layout is not touched this stage; the helpers are dead code until Stage 2 wires them into ``_build_layout``. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat(dx): wire boot-log region into the Rich Live panel (stage 2 of #394) ``LiveRunDisplay._build_layout`` now adds an optional ``boot_log`` region between the services widget and ``main`` during the docker startup window. Activation gate: ``_total_trials == 0`` AND ``_docker_boot_records(_log_buffer)`` non-empty. The region collapses the moment trials dispatch, mirroring the services-widget collapse. Height is granted from a bounded budget so the stable-height invariant (``sum(top-level Layout.size) == max(12, viewport - 1)``) holds in every state — including small viewports where a naive ``main_h = max(5, total - …)`` would silently overflow ``total`` and re-anchor Rich Live (the #392 stacking regression). ``budget = total - services_h - bottom_h - 5``; ``boot_log_h = min(desired, budget)``; below three rows the region is dropped entirely (a bordered panel needs ≥ 3 rows for one content line). ``main_h`` then reclaims the remainder without ever dropping below its floor of 5. The render is called with ``max_lines=boot_log_h - 2`` so the Panel's own content-row budget matches the region's granted rows exactly. Because ``_render_boot_log_tail`` keeps the LAST ``max_lines`` records, the newest milestones survive the clamp — hard-coding ``max_lines=5`` would inverse the "most-recent-last" contract on clamped small terminals (Rich crops a taller Panel from the bottom). Introduces ``_BOOT_LOG_MAX_LINES = 5`` so the helper default and the layout's desired height share one source. Unit tests lock the three activation states (present / absent-on-run- start / absent-when-no-docker-records) and three height regimes (no-clamp / clamp-but-present / drop), asserting the row-sum invariant and, for the clamped regime, that the rendered text contains the last two records and not the older ones (the crop-inversion regression). The four pre-existing goldens replay ``run_started(total_trials=50)`` so the region is never active — they remain byte-identical. Docs: ``docs/CLI.md`` § Live run panel rewrites the intro sentence (``three conditional ones``) and adds a bullet for the Boot log region; ``CHANGELOG.md`` gains a matching ``### Feat`` entry under ``## Unreleased``. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * test(canonical): golden SVGs for the boot-log panel region (stage 3 of #394) Add two byte-level SVG goldens capturing the Rich panel during the boot window — before ``run_started`` fires — with the "Boot log" region populated. Widths 80 and 120. The fixed record sequence includes ``tolokaforge.docker.wait_for_services`` (17 chars, the widest realistic docker short-name) alongside ``stack`` / ``builder`` / ``container`` records so the goldens lock the short-name column width. Real ``logging.LogRecord``s with pinned ``created`` / ``msecs`` keep the ``HH:MM:SS.mmm`` bytes deterministic; the helper already renders ``record.created`` in UTC so no ``TZ`` / ``tzset`` pin is needed. The recorder ``Console`` installs the shared ``THEME`` so ``border_style="muted"`` on the boot-log panel and the phase-spinner bottom bar resolve. The four pre-existing goldens are byte-untouched. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * chore(dx): tighten boot-log layout math + docstring nits (reviewer) Reviewer nits from PR feedback on the boot-log widget: - ``_build_layout`` now subtracts ``banner_h`` from the boot-log budget so the row-sum invariant holds even in the contrived state where ``_banner`` and ``_log_buffer`` are both populated at ``_total_trials == 0``. No current emitter fires both regions simultaneously, but the math should not silently depend on that caller-ordering invariant — a future emitter that raised ``_banner`` before dispatch would otherwise re-introduce the #392 stacking regression. - ``_render_boot_log_tail``'s docstring drops the misleading "callers must guard the empty-filtered case" line. Its internal ``_docker_boot_records`` call is idempotent, so pre-filtered and unfiltered inputs both work; the current call site pre-filters for activation-gate reasons unrelated to correctness. Adds ``test_boot_log_layout_self_consistent_when_banner_and_boot_log_co_occur`` locking the row-sum invariant at viewport 20 (the tightest regime where the OLD math overflows by exactly ``banner_h == 5``). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Absorbs the RunDisplayEvents seam extraction (#416) plus main-side work that landed in parallel (automation pipeline, per-service log capture, grading merge fix, shared-stack capability advertisement fix). feat/ terminal-dx retains its milestone-11 CLI + panel + Textual TUI stack on top of the now-extracted engine seam. Conflict resolution: - tolokaforge/core/* — --ours (feat/terminal-dx = superset of main events wiring + CLI-side plumbing). - tests/unit/test_run_display_wiring.py — --theirs (main = falsification- proven post-#416 version, 840 lines vs feat's 524). - CHANGELOG.md — hand-merged Unreleased section; both branches' Feat/Fix entries preserved. - AGENTS.md, docs/architecture/adr/README.md, pyproject.toml — hand- merged union. Enables #389 (seam extension) to target main cleanly and #391 (Rich + Textual consumers) to target this branch on top of the extracted seam. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…drops The previous merge (e832987) used `git checkout --ours` on the engine files, silently discarding main-side additions from #301 (network policy enforcement) and #302 (per-service log capture on trial failure) while preserving the seam wiring from #416. This commit restores those main-side symbols alongside the seam wiring: - compose_materialisation.py: NETPOLICY_EDGE_NETWORK / NETPOLICY_INTERNAL_NETWORK constants, NetworkPolicyError, enforce_network_policy, apply_network_policy_to_ compose_file, LogCaptureConfig, capture_compose_service_logs, trial_services_dir, write_capture_manifest, verify_network_policy_supported. - orchestrator.py: ComputeConfig + LogCaptureConfig imports, _build_log_capture, _project_seed_registry, _select_backend_from_tasks, _resolve_effective_runtime_choice, task-driven backend selection in _construct_runtime_backend, requires_per_trial + ephemeral-service checks in _verify_isolation_compatibility, project_loader.resolve() call in _extract_run_env_manifest, seeds + log_capture threaded to backends and executor. - per_trial_runtime.py: network-policy enforcement + reset-recipe dispatch + capture_service_logs + provision-failure log capture, seeds registry, advertised_capabilities, service_names on the env handle. - shared_stack_runtime.py: network-policy enforcement, seeds registry, advertised_capabilities, capture_service_logs no-op documented. - trial_executor.py: log_capture ctor param, _capture_service_logs and _amend_metrics_with_captured_logs threaded after conductor.run. - runtime.py: capture_service_logs on the Protocol + InMemoryRuntimeBackend + call log; advertised_capabilities on the Protocol + fixture; ProvisionStage extended with "reset_recipe". Verification: unit + canonical tiers now collect (previously 4 ImportError collection failures) and all 3414 tests pass on both tiers. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…tity (#391) (#459) * feat(dx): Rich panel consumes llm_call trio + model identity (#391) The Rich Live panel now surfaces in-flight LLM state on the focused trial. `_TrialCard` carries the four wire-state fields (llm_role / llm_provider_model / llm_call_start_ts / llm_retry_state) plus the per-role model identity (agent_model / user_model) supplied by the widened `trial_started`. The right pane opens with a `model:` header when the orchestrator names the agent's model and appends a `⏳ waiting on {role}: {provider}/{model} — {elapsed:.1f}s` line while an attempt is on the wire or a `↻ retry {attempt}/5 after {n}s ({reason})` line while the outer-retry hook is scheduling the next attempt. Both lines clear on `llm_call_finished` and on trial-terminal transitions so the pane never carries stale wire state. Stage 1 of the consumer PR — the Rich panel only. The Textual TUI gains no-op stubs for the three new methods (full state tracking + Overview "Live calls" section land in Stage 2). Existing SVG goldens stay byte-identical: replay sequences that emit no `llm_call_*` events or `agent_model` render exactly as before. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat(dx): Textual TUI consumes llm_call trio + model identity (#391) The Textual TUI's focused pane now surfaces the same in-flight LLM state the Rich panel does: a `model: <name>` header when `trial_started` supplies `agent_model`, plus a `⏳ waiting on {role}: {provider}/{model} — {elapsed:.1f}s` line while an attempt is on the wire or a `↻ retry {attempt}/5 after {n}s ({reason})` line while the outer-retry hook is scheduling. Both lines clear on `llm_call_finished` and on trial-terminal transitions. The Overview tab gains a "Live calls" section that lists in-flight calls across all running trials — capped at 10 rows with an `…and N more` tail so the tab stays scannable under high concurrency. Stage 2 of the consumer PR. `TextualRunApp.trial_started` also widens to accept `agent_model` / `user_model` kwargs, closing the post-rebase gap where the orchestrator would `TypeError` calling the narrow TUI signature. `_format_call_state_line` refactors to individual field kwargs so the Rich panel and the TUI's focused pane + Overview section share one renderer. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * hygiene(tests): relocate 20 CLI/banner test files to tests/unit/dx/ Files that import from tolokaforge.dx.* belong under tests/unit/dx/ so a headless install (`pip install tolokaforge`, no [dx] extras) can collect and run the engine test tier without pulling the dx surface. Reinforces ADR-0019's front-end plugin boundary at the test tree. Moved: - 6 files originally listed in #456: test_cli_stdout_contract, test_cli_model_cost_config, test_cli_budget_flags, test_logging_cli_flags, test_cli_status, test_run_display_cost_meter. - 14 additional CLI/banner test files surfaced by a broader audit: test_cli_display_flag, test_cli_display, test_cli_grouped_help, test_cli_assets, test_cli_resume, test_cli_commands, test_cli_fallback_flag, test_cli_budget_integration, test_dry_run_cli, test_adapter_convert_cli, test_validate_rubric_migration, test_run_banner, test_run_banner_cli_integration, test_run_banner_stopped. One inter-test import path updated (test_cli_display_flag.py imports _make_stub_orchestrator from test_cli_stdout_contract; both moved together, so the import updates from tests.unit.* → tests.unit.dx.*). Remaining coupling — deliberate cross-cutting tests staying under tests/unit/ pending a per-file split (out of scope here): - test_run_display.py — mixes Protocol tests (engine) with LiveRunDisplay tests (dx). Needs a split. - test_run_display_cli_integration.py — integration test spanning engine + dx. - test_logging_formatter.py — mostly engine, one THEME reference. - tests/unit/llm/test_preset_overlay_worker_propagation.py — engine test that imports a dx helper; the helper should probably move to core in a future refactor. 3458 unit+canonical tests still pass — pure relocation, zero behaviour change. Closes the "engine + dx test-tree decoupling" follow-up (#456's scope minus the four cross-cutting files noted above). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(dx): apply reviewer corrections to observability consumers - tui.py: cap allocation in ``_snapshot_live_calls_locked`` — return ``(visible, total)`` so only the first ``_LIVE_CALLS_MAX_ROWS`` lines are formatted, while ``total`` still drives the "…and N more" tail. Bounds formatting cost at O(rows) instead of O(in-flight trials). - test_run_display.py: drop issue-number attribution from section header; rephrase focused-pane test docstring as a forward-looking invariant. - test_cli_display_flag.py: fix stale module path in docstring. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Contributor
|
Claude encountered an error after 0s —— View job I'll analyze this and get back to you. |
click-repl 0.3.0 reserves the `:` prefix for internal commands, so bare `help` / `exit` fall through to click's dispatcher and die with "no such command". Banner now advertises `:help` / `:exit` (matching what click-repl actually accepts) plus notes Ctrl-D. Subcommand hint kept explicit. Locked by test_repl_banner_names_actual_click_repl_commands so the mismatch can't silently regress. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Contributor
|
Claude encountered an error after 0s —— View job I'll analyze this and get back to you. |
6 tasks
On a TTY with no explicit --display / TOLOKAFORGE_DISPLAY / CI signal, select_display_mode() now returns DisplayMode.FULL (the Textual TUI) instead of DisplayMode.RICH. Every escape hatch survives — the CLI callback still transparently falls back to RICH with a WARNING when textual is not installed (headless / `pip install tolokaforge` without `[dx]` extras). Operators who prefer the Rich Live panel because they want the run visible in terminal scrollback after exit (Textual's alt-screen buffer clears on exit) can force it with `--display=rich` or `TOLOKAFORGE_DISPLAY=rich`. Rationale: the Textual TUI carries the enhanced-terminal features this milestone shipped — tabbed views (Overview / Logs / Services / Infra / Errors), keyboard navigation, Live-calls table, modal help screen. Making it the default surface exposes those to every operator by default instead of hiding them behind an explicit flag. Trade-off (documented in docs/CLI.md): Textual's alt-screen takeover means terminal scrollback shows nothing from the run after exit, and startup latency is ~1-2s higher. Rich remains one env-var away for operators who want the old shape. Two existing tests updated to assert the new default; behaviour for every explicit --display / TOLOKAFORGE_DISPLAY value is unchanged. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Contributor
|
Claude encountered an error after 0s —— View job I'll analyze this and get back to you. |
…ing redesign The full-as-default flip (0d36385) exposed a pre-existing bug in TextualRunApp: Textual's LinuxDriver calls signal.signal(SIGTSTP) at startup, which Python restricts to the main thread — but TextualRunApp.__enter__ spawns Textual on a daemon thread. The failure went undetected because the Pilot-driven unit tests use App.run_test() (headless, no driver, no signals). A real --display=full invocation crashes with `ValueError: signal only works in main thread of the main interpreter`. Reverting the default so `tolokaforge run` works out of the box on a TTY. The TUI still ships behind the explicit --display=full flag — the same runtime crash remains until the threading design is fixed. Fix requires either running Textual on the main thread and hoisting the orchestrator onto a worker thread, or installing a signals-free driver via App(driver_class=...). Follow-up (I'll file separately): the underlying threading bug is independent of the default-selection question. Once fixed, we can re-flip. Two tests reverted alongside; behaviour for every explicit --display / TOLOKAFORGE_DISPLAY value is unchanged. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…470) Textual's LinuxDriver installs SIGTSTP / SIGCONT (in __init__) and SIGWINCH (in start_application_mode) handlers. Python restricts signal-handler installation to the main thread, and TextualRunApp runs the driver on a daemon thread — so the TUI crashed before mount. Subclass LinuxDriver and wrap the three signal-installing entry points in a scoped monkey-patch that downgrades the specific "signal only works in main thread ..." ValueError to a no-op. Ctrl-Z suspend and mid-run terminal-resize reflow become no-ops — accepted trade-off documented in docs/CLI.md § Full TUI. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Textual's App.__init__ does `self.driver_class = driver_class or self.get_driver_class()` at app.py:637 — the class-level `driver_class = _SignalTolerantLinuxDriver` attribute on TextualRunApp was silently shadowed by the platform-detected LinuxDriver. Traceback from a real invocation confirmed the base LinuxDriver was still being instantiated: `self = <LinuxDriver …>` at linux_driver.py:78. Fix: override the get_driver_class() method (Textual's documented extension point) to return our subclass on POSIX. Falls back to super() on Windows. Test updated to assert the actual behaviour — instantiate the app and check the resolved driver class — rather than checking a class attribute that Textual explicitly overwrites. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The Textual TUI has been withdrawn as an operator surface: three threading-model iterations (signals-free driver, get_driver_class override, background-thread orchestration) still left it "weird" enough that no more upstream-Textual workarounds are going to land. Rip it out in one commit — module, tests, docs, ADR wording, dep. Compatibility surface break (deliberate, no deprecation cycle): - --display=full now exits 2 with "Invalid value for '--display': 'full' is not one of 'rich', 'plain', 'log', 'none'." — scripts passing the flag must switch to --display=rich or drop it. - TOLOKAFORGE_DISPLAY=full exits 2 with "invalid TOLOKAFORGE_DISPLAY value 'full'; expected one of: rich, plain, log, none." — stale exports must switch to a valid value or `unset`. Removed: - tolokaforge/dx/tui.py (whole module) and tests/unit/dx/test_tui_app.py. - DisplayMode.FULL, _textual_available(), the FULL branch of LiveRunDisplay.for_mode, and the Textual fallback in _resolve_display_mode. - textual>=0.85.0 from [project.optional-dependencies].dx and [dependency-groups].dev. uv.lock relocked; `python -c "import textual"` fails with ModuleNotFoundError after `uv sync --dev`. Docs and ADR: - docs/CLI.md: deleted the "Full TUI" section, updated the display-modes table and the precedence, banner display-mode table, and dry-run composition to the four remaining values. - docs/architecture/adr/0019-front-end-plugin-namespace.md: LiveRunDisplay is now the sole reference RunDisplayEvents consumer; the Protocol boundary itself is unchanged. Tests: - Added test_display_mode_enum_declares_four_values, test_no_textual_module_in_dx_extras (parse pyproject.toml), test_explicit_full_flag_is_rejected_as_invalid_choice, and test_tolokaforge_display_full_env_var_is_rejected. - Removed every DisplayMode.FULL / TextualRunApp / _textual_available test. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
`tolokaforge run --display=rich` opens a POSIX cbreak keyboard listener on `LiveRunDisplay.__enter__` so operators can page between trials during a run. Focus starts in auto-follow (byte-identical to today's behaviour) and flips to manual on the first nav key: - `j` / `k` — focus next / previous visible trial in `_visible_cards()` order (the same order the left pane shows). - `H` / `L` — jump to first / last visible trial. - `f` — toggle auto-follow; when re-enabled focus snaps to the card with the newest `last_update_ts` (the card auto-follow would have picked on the most-recent lifecycle event). Manual mode gates the four existing focus writers (`trial_started`, `trial_completed`, `trial_failed`, `judgment_scored`) via a new `self._auto_follow: bool` on `LiveRunDisplay` — new lifecycle events still re-order the pane, but focus stays where the operator put it. The bottom bar prepends `[j/k nav · f follow] ` (escape()'d so Rich does not consume the bracketed prefix as an unknown style tag) whenever manual mode is active and at least one trial has started. The listener short-circuits — no thread started, no termios mutation — when any of three guards trips: `sys.stdin.isatty()` False (piped stdin, CI), `sys.platform == "win32"` (different terminal-input model), or `TOLOKAFORGE_INTERACTIVE_PANEL=0` (explicit escape hatch). Termios is captured on `__enter__` and restored under `try / finally` on `__exit__`, so an in-run exception still leaves the terminal usable. Cbreak (not raw) mode preserves `Ctrl-C`. Tests (10 new, real behaviour — no listener mocks): - `test_keyboard_listener_no_ops_when_stdin_not_a_tty` — non-TTY stdin short-circuits `_should_start`; focus behaviour unchanged. - `test_env_var_zero_disables_listener` — `TOLOKAFORGE_INTERACTIVE_PANEL=0` keeps the panel in auto-follow-only mode on a TTY-shaped stdin. - `test_j_focuses_next_visible_trial_and_disables_auto_follow` — three trials, `j` moves from newest to next-newest and flips `_auto_follow`. - `test_k_focuses_previous_visible_trial` — parametric inverse. - `test_H_jumps_to_first_visible_and_L_to_last` — parametrised over (`H`, first-in-visible-order) / (`L`, last-in-visible-order). - `test_f_toggles_auto_follow_and_reasserts_current_leader` — after a manual `j`, `f` restores auto-follow AND snaps focus to the newest. - `test_manual_mode_suspends_auto_follow_on_new_event` — after `j`, `trial_completed` on a different trial does NOT steal focus. - `test_listener_restores_termios_on_exit_even_after_exception` — real `pty.openpty()` fd; forces an exception inside the `with`, asserts ICANON is restored + every non-driver-managed termios field matches the pre-cbreak snapshot (macOS PTYs stamp the driver-only `EXTPROC` bit after cbreak, so it is masked from the comparison). - `test_bottom_bar_hint_appears_in_manual_mode_and_hides_in_auto_follow` — asserts the `[j/k nav · f follow]` substring under both states. Full unit + canonical suite: 3440 passed (2965 unit baseline + 10 new + 465 canonical). Existing SVG goldens are byte-identical — replay never enters manual mode so the bottom-bar `default` cost-style path still returns a plain `Text(...)`. Docs: docs/CLI.md `§ Keyboard navigation` (under the Live-run-panel section) documents the bindings, the three guard cases, and the `TOLOKAFORGE_INTERACTIVE_PANEL=0` escape hatch. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Show `Focused trial · <task_id> · <trial_index>` instead of a bare "Focused trial" heading so `j`/`k` nav has visible signal about which trial the right pane is showing. Waiting-for-first-trial state keeps the bare title.
Show `Focused trial · N/M` where N is the focused trial's 1-based position in the visible-cards nav order and M is the visible-cards count. Task id + trial index remain visible inside the pane body on the failure path; the header now stays scannable at a glance.
Press `l` in the Rich Live panel to swap the Focused-trial pane body between its summary and a stream of log records emitted during that trial's execution. Logs are scoped to the focused trial via a run-time `contextvars.ContextVar` set by the runner around each trial and stamped onto records by the panel's log sink; records that already carry a `trial_id` (Docker logging pipeline) are preserved. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Read raw bytes from the terminal fd with os.read(fd, 32) instead of the buffered self._stdin.read(1), so burst input (fast typist, paste, or a multi-byte ESC sequence) no longer strands keystrokes in a userspace buffer invisible to select(). Parse arrow keys and Home/End ESC sequences (CSI and SS3 variants) into their nav keys via a small state machine; drop unknown, over-long, and bare-ESC sequences instead of dispatching garbage. EOF on the read fd now exits the thread cleanly. Surface the H/L and arrow bindings in the manual-mode bottom-bar hint, show a distinct placeholder when the log pane is toggled on before the first trial, and degrade to auto-follow-only when the cbreak setup raises rather than aborting the Live setup. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The formatter conflict on `tests/unit/test_run_display.py:560` had persisted through the milestone-17 branch (AGENTS.md gotchas #3/#4); switching the failing `assert (…), "…"` to a lift-message-to-variable shape satisfies both formatters unambiguously. The six `tests/canonical/golden/run_display/*.svg` goldens drifted against the intentional title rename in `a876510` (`Focused trial` → `Focused trial · N/M`). Regenerated via `pytest … --update-canon`; byte diff is exclusively the new position/total suffix in the pane header.
CiroGamboa
changed the title
Contributor
|
Claude encountered an error after 1s —— View job I'll analyze this and get back to you. |
# Conflicts: # CHANGELOG.md # README.md # docs/ARCHITECTURE.md # docs/OUTPUT_FORMAT.md # docs/adr/0019-front-end-plugin-namespace.md # tolokaforge/core/orchestrator.py # tolokaforge/dx/cli/main.py
Terminal-dx's stdout=artifact / stderr=progress contract (from #280) routes `console.print` output to stderr. Main-side commit e1744f5 (#494) added `test_convert_validate_exits_nonzero_for_invalid_output` asserting on `result.output` (stdout), which surfaced as a failure after the main-merge. Assert on `result.stderr` instead — matches every other test in this file that inspects `console.print` output.
CiroGamboa
force-pushed
the
feat/terminal-dx
branch
from
July 21, 2026 15:34
fde0362 to
508e526
Compare
Contributor
|
Claude encountered an error after 0s —— View job I'll analyze this and get back to you. |
The Textual TUI was removed in fca2b53. The log-tail widget landed in the Rich panel itself (per-trial log stream in the focused pane) — update the LiveRunDisplay.log_records docstring assertion to name the actual consumer.
Contributor
|
Claude encountered an error after 0s —— View job I'll analyze this and get back to you. |
…l TUI The five per-issue implementation memos under docs/plans/ described the Rich panel work with `--display=full` and the Textual TUI as still-planned follow-ups (milestone C3). That work was tried, abandoned, and removed in fca2b53. The memos have served their purpose and now misdescribe the shipped shape — delete them. Commit history preserves the design record.
Contributor
|
Claude encountered an error after 0s —— View job I'll analyze this and get back to you. |
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
|
|
||
| stream = _FakeStream(is_tty=False) | ||
| configure_root_logging(log_format=LogFormat.PLAIN, stream=stream) | ||
| logging.getLogger("t").warning("token %s in header", secret) |
6 tasks
Adds a transport-agnostic Components model as a first-class engine
concept alongside the existing RunDisplayEvents surface, so front-ends
can render one calm status board over infrastructure of arbitrary shape
(docker services, gRPC clients, per-trial containers, future k8s pods).
New types:
- ComponentKind — Literal of transport-agnostic labels.
- ComponentPhase — Literal lifecycle enum; {degraded, unhealthy, dead}
trigger the front-end's auto-expand-on-fail behaviour.
- ComponentSnapshot — TypedDict wire shape (id, kind, phase, detail, owner).
- build_component_id(namespace, kind, instance) — canonical id builder.
New Protocol methods (kwarg-only, no-op defaults on _NullRunDisplayEvents):
- component_registered(snapshot=…)
- component_status_changed(snapshot=…) — same id updates the row in place
- component_log_appended(component_id=…, level=…, message=…, ts=…)
- component_unregistered(component_id=…)
The four methods keep component chatter out of the panel's general log
ring: per-attempt polling loops fire component_status_changed with a
fresh detail (no scroll); failure-time log context lands in a
per-component tail buffer that only surfaces beneath an unhealthy row.
Zero break for existing callers: methods are defaulted no-ops via
structural typing; existing tests updated to reflect the widened surface
(12 → 16 declared methods).
Design record: ADR-0021.
The Rich Live panel gains a Components widget that renders the four new
RunDisplayEvents component_* events as a compact status board — one row
per registered id, updated in place on status changes, with a small log
tail auto-expanding beneath any row whose phase is degraded / unhealthy /
dead. Widget visibility rule: on when any component is tracked AND
either the run is in its startup window OR at least one component is
currently unhealthy. Healthy post-startup runs hide the widget; a
mid-run infrastructure failure re-surfaces it automatically.
Legacy paths kept working via adapter shims:
- _service_to_component lifts each phase_changed(services=[…]) row into
a ComponentSnapshot under owner="engine".
- _container_to_component lifts each trial_provisioned(containers=[…])
row into a ComponentSnapshot under owner=f"trial/{trial_id}".
Reference reporter — GrpcRunnerClient.connect() in shared_stack_runtime.py:
the retry loop used to log one INFO line per attempt ("Waiting for
Runner service (attempt N, elapsed=…)"), which read as noise in the
panel and above it. It now fires component_registered on entry, then
component_status_changed with a fresh detail per attempt (SAME row,
no scroll), then a terminal healthy / unhealthy transition. The
per-attempt logger.info is downgraded to DEBUG so `-v` still surfaces
attempts for diagnosis. SharedStackRuntimeBackend.__init__ threads
events= through to the client; Orchestrator._create_runtime_backend
passes self._events.
The layout region formerly named "services" is renamed "components" and
the Panel title changes from "Services" to "Components"; layout
invariants (stable-height clamp, boot-log co-existence) are preserved.
- ADR-0021 (Component-oriented monitoring for tolokaforge run): design
record for the seam, namespace convention, reporter contract, layout
visibility rule, and the backwards-compat story via adapter shims.
- docs/CLI.md § Live run panel: rewrites the Services bullet as the
Components widget with phase icons, auto-expand-on-fail description,
and a pointer to ADR-0021.
- docs/RUNTIME_BACKENDS.md § Reporting component status: reporter-author
guide with a worked example and namespace guidelines.
- tests/unit/test_run_display.py: 6 new tests covering
registered/status_changed upsert, log-tail ring bound, mid-run
visibility on unhealthy components, tail auto-expansion beneath
unhealthy rows, unregister semantics, and GrpcRunnerClient's
component events on the healthy path.
- tests/unit/test_run_display_events.py + test_run_display_wiring.py:
update the Protocol method-count assertions (12 → 16) and extend the
RecordingEvents test double with the four new methods.
- tests/unit/test_shared_stack_runtime_env_manifest.py: relax the
GrpcRunnerClient construction assertion to accept the new events=
kwarg alongside runner_address.
- tests/canonical/golden/run_display/panel_boot_log_{80,120}.svg:
regenerated for the "Services" → "Components" title rename; no other
visible drift.
The `run` subcommand accumulated 8 new flags across M11. Deep review found two straight-up overlaps with existing config, two flags whose per-invocation override utility was thin, and one modifier for a default that's essentially never tuned. Result: `run --help` drops from 17 to 13 flags without losing any feature. **Cut (config-side keeps the feature):** - `--sample-limit` — pure removal. The overlap with `orchestrator.repeats` × `evaluation.tasks_glob` is enough for the smoke-test use case. `SampleBudget` and `make_budget(sample_limit=)` stay for programmatic callers. - `--model-cost-config <path>` → `observability.pricing_overlay_path:` in the run config. `ObservabilityConfig` gains one `Path | str | None` field; the CLI calls `pricing.reload_pricing(overlay_path=...)` after parsing the config, before the orchestrator is built. - `--fallback-models m1,m2,...` → `models.agent.fallbacks: [...]` in the run config. `ModelConfig` gains a `fallbacks: list[ModelConfig]` field. The `FallbackLLMClient` wiring is unchanged; the CLI reads the chain from `run_config.models["agent"].fallbacks` instead of a comma-separated CLI string. `_parse_fallback_models` is dropped. **Cut (default hardcoded):** - `--dry-run-samples N` — removed. `_DEFAULT_DRY_RUN_SAMPLES = 3` is now a module constant; three preview panels is plenty for spotting a misconfigured prompt. Packs smaller than three render every task they have. Tests: `test_cli_budget_flags.py` narrows to cost + time; the "all three limits produce a three-child composite" test becomes "both limits produce a two-child composite". `test_cli_fallback_flag.py` and `test_cli_model_cost_config.py` rewritten to exercise the config-field paths. `test_dry_run_cli.py` drops the samples-flag test. Docs: `docs/CLI.md § Cost, time, and sample limits` → `§ Cost and time limits`; `§ Fallback models` and `§ Custom pricing overlay` rewritten to show the YAML shape. `docs/OUTPUT_FORMAT.md` reflects the two-way LIMIT_HIT.json `which` field.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Improved Terminal Developer Experience
TL;DR
tolokaforge runused to scroll a wall of log lines. This PR turns it into a Rich Live progress panel with per-trial cards, keyboard-navigable focus, a live per-trial log stream, cost/time/sample budgets, dry-run preview, resume, a dashboard-URL banner, structured logging withpretty/plain/jsonformats, and a--display={rich,plain,log,none}toggle so the same binary works on a laptop TTY, in CI, in a script pipeline, and in a headless server install.All new UI code lives in a separate
tolokaforge.dxnamespace behind the optional[dx]extras. The engine talks to the front-end through one Protocol seam —RunDisplayEvents— with a no-op default. Headless installs (pip install tolokaforge, no extras) are byte-identical to pre-M11 behaviour. The front-end can be swapped, sub-classed, or removed without touching the engine.Zero task-authoring surface changes; every
task.yamlin the tree loads unmodified.Closes #392, #394, #391. Also carries the merged squash-commits from M11 (#285, #279, #280, #281, #282, #283, #284, #286, #278) and M11.1 (#392, #394) as they landed on
feat/terminal-dx.What ships
1. Rich Live progress panel —
--display=rich(the default on a TTY)Three-region layout that redraws in place at 4 fps:
⏳running,✓completed,✗failed) with cost, tokens, turn count, last event.Tab/j/k/↑/↓/ arrow keys navigate;H/Ljump to first/last;ftoggles auto-follow (the panel tracks the newest event unless you've pinned focus manually).completed/total · running · $cost · in X / out Y tok · fail N · eta, plus a cost-budget meter that goes amber at 80 % and red at 100 % of--cost-limit.Two variable-height widgets sit inside the panel with a stable-height clamp (they steal rows from the trial list, never lengthen the panel):
tolokaforge.docker.*INFO records renderedHH:MM:SS.mmm | short-name | message. Disappears when trials dispatch.lon a focused trial to swap the summary pane for a live tail of that trial's log records, auto-tagged via a run-time context variable so records emitted outside any trial's execution never leak in.2.
--displaytoggle — pick your terminal experienceFour modes; the group callback resolves the flag once and stashes the resolved enum on
ctx.obj["display_mode"]so subcommands don't re-parse:richplainlognonePrecedence: explicit
--display=…>TOLOKAFORGE_DISPLAY=…env var >CIenv var truthy →plain>sys.stderr.isatty()→rich>plain.TOLOKAFORGE_INTERACTIVE_PANEL=0is an escape hatch that disables the cbreak keyboard listener without changing the render — panels still redraw, they just don't respond to keys. For CI, exotic ptys, and operators who prefer the pre-listener behaviour.3. Interactive REPL —
tolokaforgewith no subcommandDrops into a
click-replshell with tab-completion of every subcommand, flag, and file path. Command history at~/.tolokaforge_history. Exit withexit,quit, or Ctrl-D. Root flags (-v,-q,--display,--log-format) apply for the session's lifetime.4.
tolokaforge run --dry-run [--dry-run-samples N]Resolves the config and adapter with full parity to a real run, then renders the first N samples as Rich panels — system prompt, first user prompt, sanitized tool spec, resolved agent / judge / runtime identifiers — and exits 0 without any provider HTTP call. Two layers of defence: patched
httpx.Client.sendsentinel + patchedlitellm.completionbindings; both trip anAssertionErrorif the code path ever tries to hit the network.5.
tolokaforge run --resume— pick up where a run stoppedResolves the existing run dir (fixing the pre-milestone bug where every resume allocated a fresh timestamped dir), reads
run_state.json, and re-runs only the pending / infrastructure-failed trials. Idempotent on a fully-complete run ("Nothing to do; run already complete"). Worker restart on a populated queue (tolokaforge worker --run-dir <existing>) resumes automatically via the durable queue.6. Cost / time / sample budgets —
--cost-limit,--time-limit,--sample-limitAny budget hit triggers graceful shutdown: in-flight trials finish,
LIMIT_HIT.jsonlands under the run dir naming the hit reason, and the end banner shows⏸ Run stopped (<reason>). The Rich panel's cost segment turns amber at 80 % and red at 100 % of--cost-limit.--time-limitaccepts compound-unit spec strings (30m,2h,1h30m).--sample-limitcaps trial count.7.
--fallback-modelsOrdered per-generate cursor letting a batch survive provider outages. Wrapped as a
FallbackLLMClientand injected viaOrchestratorDeps.agent_client_factory— no orchestrator-side plumbing needed.8.
--model-cost-configOverlays JSON/YAML onto the shipped pricing table before the orchestrator is built, so every downstream cost computation (including litellm's fallback path) sees the merged rates.
9. Dashboard-URL banner around every run
Two-line start banner (
→ Run: <run-id>+→ Report: file:///…/) before the run; three-line end banner (✓ Run complete in <duration>or✗ Run failed, plus the report path and→ Browse: tolokaforge browse <run-id>) after — including on failure. URLs are OSC 8 hyperlinks. Silenced under--display=none.10. Structured logging —
-v,-q,--log-format={pretty,plain,json}Format is
HH:MM:SS.mmm | LEVEL | k=v | message. Auto-selectsprettyon TTY andplainon pipe.--log-format=jsonfor machine consumers. The palette matches the shared theme.11. Root help layout +
--versiontolokaforge --versionprints the installed package version. Root--helpgroups commands under Runs / Tasks / Docker / Config / Assets / Adapters headings, alphabetical within each.12.
stdout= artifact-path,stderr= progresstolokaforge runandtolokaforge prepareemit the absolute run-dir path as a single line onsys.stdouton success. Every other command leaves stdout empty. Idiom:RUN_DIR=$(tolokaforge run --config …). Rich panels, logs, and banners all route to stderr.Design — how the pieces fit
flowchart TD subgraph "Engine (tolokaforge.core)" seam[RunDisplayEvents Protocol<br/>12 kwarg-only methods<br/>_NullRunDisplayEvents no-op default] orch[Orchestrator.run] --> seam cond[Conductor._grade] --> seam exec[ProvisioningTrialExecutor] --> seam runner[TrialRunner + _AgentMetricsSink] --> seam client[LLMClient.generate<br/>Retrying controller] --> seam end subgraph "Front-end plug-in (tolokaforge.dx — optional extras)" panel[LiveRunDisplay<br/>Rich panel + keyboard + widgets] banners[start / end banners] replmod[click-repl] dryrun[dry_run_render] end engine[Engine call sites] seam -.-> engine engine --> panel engine --> bannersThe seam
tolokaforge/core/run_display_events.pypublishes a single Protocol:Every method is kwarg-only and every new field on a widened method is defaulted, so out-of-tree implementers never see a positional-caller break.
Pluggability
RunDisplayEventsreference. The default is_NullRunDisplayEvents— every method is a no-op.LiveRunDisplayintolokaforge/dx/live_panel.pyimplements the Protocol. The CLI threadsdisplay.eventsintoOrchestratorDeps.eventswhen--display=richis active.tolokaforge.dxnamespace as the plugin surface. Recorded underdocs/adr/0019-front-end-plugin-namespace.md.Unpluggability
pip install tolokaforge(no[dx]extras) does not pull Rich,click-repl, orprompt-toolkit. Thetolokaforgeconsole script degrades to a stdlib-only shim (tolokaforge._entry:main) that prints the install hint._NULL_EVENTS— headless behaviour is byte-identical to pre-M11.--display=nonesilences the sharedconsoleand raises the tolokaforge root log handler above CRITICAL so no log record emits.--log-formatis retained for shape but has no observable effect on the success path._LogSinkto the wrapped stream regardless of display mode.Layout invariants under the Rich Live panel
Rich Live redraws in place only when the total renderable height is constant across refreshes. Any optional region (banner, services widget, boot-log widget, focused pane) steals rows from
mainrather than lengthening the panel. Locked by a dedicated test at viewport 15 assertingmain_h == 5._render_boot_log_tail(records, max_lines=boot_log_h - 2)derives its crop from the granted region height so Rich never has to trim the panel from the bottom (which would drop the newest records under Rich's default policy). Locked by aConsole.export_text()assertion.The stream-identity handler sweep
LiveRunDisplay.__enter__snapshots the four terminal-stream objects before Rich Live installs its redirect proxy, then sweeps every non-root logger inlogging.root.manager.loggerDict(skippingPlaceHolderentries) and removes anyStreamHandlerbound to one of those captured streams. This closes the channel litellm's privateLiteLLM/LiteLLM Router/LiteLLM ProxyStreamHandlers used to bypass Rich Live's cursor coordination.__exit__restores every removed handler.The sweep uses stream identity, not logger name, so we don't hard-code third-party internals (AGENTS.md Core Rule 6).
What was tried and dropped
A Textual TUI (
--display=full) was prototyped and shipped briefly, then removed end-to-end infca2b53— the reactive-model / worker-thread bridge could not stay in sync under our concurrent-trials threading model, and every fix regressed something else. This PR ships without it;docs/CLI.mddocuments only the four modes above. History is preserved in the branch (commits0d36385→a1543a0→da1d024→14fc406→fca2b53) as a "why we don't have this yet" record, and #395 tracks a future TUI resurrection built around the sameRunDisplayEventsseam but on a proper async runtime.Impact on existing tasks — read this first
task.yamlfield additions or renames. Existing task packs work unmodified.RunDisplayEventspositional-caller breakage — every new method is kwarg-only; every widened method's new fields are defaulted.--display=richbehavior changes for the better only: panel no longer stacks, boot log shows real docker milestones during the Docker-boot window, focused pane can render per-trial LLM-call state (from Observability: pre-call, retry, and non-agent LLM signals missing from engine events #389). Consumers who never installed[dx]extras see no behaviour change — the null sink is the default.origin/main— this branch is caught up with M8 (multi-container v1), M9 (project-layer v1 finalisation), and adapter/tools fixes. The main-catchup touch-points are limited to three test-side reconciliations (see the tail of the commit list).hygiene-reviewGitHub Action broken repo-wide — infra), chore(dx): remove or wire dead "priming_queue" phase label #399 (deadpriming_queuephase label), bug(orchestrator): _extract_run_env_manifest reads compose_file at wrong nesting on EnvironmentPatch #413 (_extract_run_env_manifestpre-existing crash on multi-service examples), Observability: judge/grader LLM calls run in the gRPC runner and never reach the RunDisplayEvents sink #415 (judge/grader observability via runner→orchestrator event channel — blocked by Observability: pre-call, retry, and non-agent LLM signals missing from engine events #389), orchestrator.run_worker emits no RunDisplayEvents (resumable/worker path is unobserved) #437 (Orchestrator.run_workeremits noRunDisplayEvents).Suggested review order
The branch is 30+ commits ahead of main. The load-bearing commits, in reading order:
099248a— Shared display layer ([terminal-dx] A1 — Shared display layer #276):Console,THEME,make_progress,make_live. The primitive that everything else composes on. Readtolokaforge/dx/_display.py.645bfec— Structured logging ([terminal-dx] A3 — Structured log format with --verbose/--quiet #279):LogFormatenum,configure_root_logging, theHH:MM:SS.mmm | LEVEL | k=v | messageshape. Readtolokaforge/core/logging.py.61d44b6— stdout/stderr contract ([terminal-dx] A4 — stdout is artifact, stderr is progress #280): sets thestdout=artifact-path,stderr=progressinvariant that [terminal-dx] B1 — Rich Live progress panel during run #285 and the tests depend on.2d9d76d—--displaytoggle ([terminal-dx] B2 — --display {full,rich,plain,log,none} #282): how the group callback resolves the mode and stashes it on ctx.aad98ac— Rich Live progress panel +RunDisplayEventsProtocol ([terminal-dx] B1 — Rich Live progress panel during run #285): the big one. Introduces the Protocol seam,LiveRunDisplay, and the panel layout. Every later panel-side change composes on this.0dc92ba/002fe1a— Dashboard-URL banner ([terminal-dx] A5 — Dashboard-URL banner (local file:// links) #281) +--versionand help layout ([terminal-dx] A2 — Root --version flag and grouped --help #278): banner path + root help polish. Small, self-contained.9b6a36b— Budgets + fallback + pricing ([terminal-dx] B3 — Cost/time/sample budgets and --fallback-models #283):CompositeBudget,FallbackLLMClient, pricing overlay. Threads throughOrchestratorDeps.36641bd— Dry-run ([terminal-dx] B4 —run --dry-runrenders first prompts and exits #284):dry_run.py+dry_run_render.py. Runs the adapter and stops before any HTTP call.a7608ac— Resume ([terminal-dx] B5 — run --resume idempotent replay #286):resolve_resume_run_directory,RunStateManager.recover_run.c366d66— Kill panel stacking (bug(dx): Rich Live panel stacks during trial execution — diagnose then fix #392): the stream-identity handler sweep. Layout invariant + regression test atmain_h == 5.e1bd105— Boot-log widget (feat(dx): boot-window log tail widget in Rich panel #394): reuses the layout invariant from bug(dx): Rich Live panel stacks during trial execution — diagnose then fix #392; adds the region-clamp pattern.eb6387d— Panel consumes llm_call trio + model identity (dx(observability): consume llm_call/retry events + model identity in Rich panel + Textual TUI #391): extends the Protocol with per-call events;LLMClient.generategainsobservation: LLMCallObservation | None. The@retry→tenacity.Retryingcontroller refactor is the load-bearing change.fca2b53— Remove Textual TUI: dead-code delete. Read only if curious about the "why we don't have this yet" answer.4df7cc5→086afd6→5e09f25— keyboard navigation (Tab,j/k/ arrows,H/L/Home/End,f,l) and the per-trial log-tail widget. Read_KeyboardListenerand_render_right_pane.c208220— SVG-golden refresh: canonical byte-stability. Fifteendocs/adr/…-anchored assertions.a41fb68— Merge origin/main: brings in project-layer v1 (M9), multi-container v1 (M8), adapter fixes. Non-conflicting except for three test-side reconciliations.508e526— Adapter-convert test asserts on.stderr: the terminal-dxstdout=artifact-pathcontract from [terminal-dx] A4 — stdout is artifact, stderr is progress #280 already routed Rich console output to stderr, but main's fix(adapter): fail conversion on invalid output #494 added a test that asserted on stdout. One-line reconciliation.Key design choices
RunDisplayEventsProtocol, front-ends implement it in a separate namespace_NULL_EVENTSdefault) and keeps Rich out of the engine dependency graph. Enables future front-ends (web dashboard, JSON emitter, OTLP consumer) without touching the engine.LiveRunDisplay.__enter__StreamHandlers (which bypassed Rich Live's cursor coordination viasys.__stderr__) without hardcoding provider names. AGENTS.md Core Rule 6.maininstead of lengthening the panel. Regression guard: dedicatedmain_h==5test at viewport 15._LogSink._log_buffer(no new engine hook)tolokaforge.docker.*INFO record fired during Docker boot. Adding aphase_changedpayload or a polling loop onEngineStack.get_status()would be redundant._render_boot_log_tail(records, max_lines=boot_log_h - 2)Layout(size=…)from the bottom, dropping the newest records. Derivingmax_linesfrom the granted region height preserves "most-recent-last" under clamp. Locked by aConsole.export_text()assertion.RuntimeBackend.get_infrastructure_snapshoton the Protocol, not the handleobservation: LLMCallObservation | NoneonLLMClient.generateLLMClientis built once per run and reused across concurrent trials; per-trialtrial_id/rolecannot live on__init__. Optional / defaulted → zero break for existing callers.@retrydecorator → per-calltenacity.Retryingcontrollerself._retry_sleepattribute serves as the single sleep-stubbing seam for both the new event tests and the migrated legacy_disable_tenacity_sleephelper.__enter__, restored undertry / finallyon__exit__Ctrl-C, so killing the run still works.LLMCallRoledeclares all four roles so the Protocol is complete; wiring the out-of-process channel is #415.Industry precedents
less,htop, and any TUI that reflows around inline output. Fix pattern (stream-identity match, no name lookup) mirrors howstructlogandlogginghandlers coordinate underredirect_stdout— the writer identity is authoritative, not the logger name.htop's process list + selected-process detail, IDE test runners. Domain-density on the left, drill-down on the right.RunDisplayEventsseam pattern ↔ OpenTelemetryTracerProvidershape — an engine-side Protocol with a null-provider default, so consumers opt in. Diverges deliberately:RunDisplayEventsis ephemeral panel telemetry, not persisted metrics; the null-sink default keeps headless runs byte-identical.stdout=artifact/stderr=progress↔ classic Unix filter idiom (ls | grep); enables shell composition (RUN_DIR=$(tolokaforge run --config …)).Verification
_total_index_by_keypopulation): temporarily removing the dict assignment caused two dedicated tests to fail with the exact[0, 0, 0]signature. Restored.tolokaforge run --config examples/native/tool_use/run_config.yaml— panel stayed anchored across 82 s / 2 trials / 2 workers; boot-log region rendered realwheel_resolverrecords; probe log recorded zero third-party (litellm) bypass writes.pip install tolokaforge(no[dx]extras) →python -m tolokaforgeprints the install hint;Orchestrator(...).run()in a pure-library context runs against_NULL_EVENTSwith byte-identical output to pre-M11.hygiene-reviewGitHub Action failed in ~500 ms on every PR in this milestone (pre-existing infra bug filed as ci(claude-review): hygiene-review action fails at init on every recent PR #398 — the action never inspected the diff before erroring). Merges proceeded on green branch-attributable lanes; ci(claude-review): hygiene-review action fails at init on every recent PR #398 tracks the fix.What's next
LLMCallRolefor completeness but do not fire events.Orchestrator.run_workerobservability. The resumable/worker path currently emits noRunDisplayEvents; pre-existing since refactor(core): extract RunDisplayEvents engine seam from feat/terminal-dx to main (M11 decoupling) #416.RunDisplayEventsseam.hygiene-reviewGitHub Action._extract_run_env_manifestcrash blockingmulti_service*examples.priming_queuephase label.Try it locally