Skip to content

Commit 515f91c

Browse files
ThomasK33Mux
andauthored
feat: Agent Terminal v1 — Week 4 design gap closure (#7)
* docs: add week 3 status and week 4 plan Update stale design docs to reflect shipped Week 3 work, add a formal Week 3 status record, and add a concrete Week 4 plan focused on the remaining design-parity gaps.\n\n---\n_Generated with [`mux`](https://github.com/coder/mux) • Model: `openai:gpt-5.4` • Thinking: `xhigh`_ * Add file input support to type and paste * Add cursor position render waits * Add shared CLI context and exit-code mapping * Align CLI integration exit codes * Add create env, term, name, and shell options * Extend session status state machine * Harden session lifecycle reconciliation * Update CLI handlers for destroyed sessions * Fix wait --wait-for-exit terminal guard order * test: cover failed and destroyed session states * Fix lint violations in lifecycle changes * test: accept terminal export status after force destroy * feat(renderer): add profile hash and snapshot metadata contracts * style: format files for prettier compliance * Add screenshot metadata to backend and CLI * Enrich export metadata headers * style: format enriched metadata files for prettier compliance * Add snapshot include-scrollback flag * Handle snapshot scrollback RPC option * Add ghostty-web snapshot scrollback support * Add snapshot scrollback test coverage * style: format files for prettier compliance * Fix snapshot test mock parameter types * style: fix formatting after WS1+WS2+WS3 integration * docs: sync Week 4 status and remaining gaps * test: add unicode-grid fixture app * test: add scrollback-demo fixture app * test: add E2E tests for unicode-grid and scrollback-demo * dogfood: add week4-cli-parity proof bundle * dogfood: add week4-unicode-review proof bundle * dogfood: add week4-scrollback-review proof bundle * dogfood: add week4-failure-recovery proof bundle * style: fix dogfood formatting * docs: mark week 4 fixtures as shipped * Add March 22 dogfood proof bundles * Add March 22 color and alt-screen dogfood artifacts * Add dogfood artifacts for unicode and scrollback scenarios * Add 2026-03-22 dogfood CLI artifacts * Handle wait render host-unreachable fallback * Preserve session creation dimensions for replay * Fix scrollback snapshot viewport normalization * Recreate renderer backend on replay resize * Fix lint errors in viewport normalization changes * Update integration tests for viewport normalization * style: fix formatting and rename mistyped dogfood files * style: rename remaining mistyped dogfood files to .txt * style: format dogfood JSON files * style: format unicode dogfood files * style: format all dogfood artifacts * style: rename all non-JSON dogfood files to .txt * docs: add 20260323 dogfood bugfix evidence * Fix review findings for input handling and docs --------- Co-authored-by: Mux <mux@local>
1 parent 8ae3bde commit 515f91c

406 files changed

Lines changed: 12171 additions & 353 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

README.md

Lines changed: 79 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -1,16 +1,85 @@
11
# agent-terminal
22

3-
Node/TypeScript CLI scaffold.
3+
`agent-terminal` is a Node/TypeScript CLI for launching, controlling, inspecting, and exporting reviewable terminal sessions.
4+
It is built for agent workflows that need both semantic state and visual artifacts from live or exited TUIs.
45

5-
## Setup
6+
## Quick start
67

7-
1. `mise install`
8-
2. `mise run bootstrap`
9-
3. `mise run ci`
8+
```bash
9+
mise install
10+
npm ci
11+
npx playwright install chromium
12+
npm run build
1013

11-
## CI
14+
SESSION_ID=$(node dist/cli/main.js create --json --name demo | jq -r '.data.sessionId')
15+
node dist/cli/main.js type "$SESSION_ID" 'echo hello from agent-terminal'
16+
node dist/cli/main.js send-keys "$SESSION_ID" Enter
17+
node dist/cli/main.js inspect "$SESSION_ID" --json
18+
node dist/cli/main.js destroy "$SESSION_ID"
19+
```
1220

13-
- GitHub Actions uses `mise` as the canonical entrypoint for tool setup and quality gates.
14-
- The committed workflow in `.github/workflows/ci.yml` is hand-curated. `mise generate github-action` is useful as a scaffold, but the checked-in file is the maintained source of truth because it includes repo-specific triggers, bootstrap behavior, and step-level logs.
15-
- CI uses `mise run bootstrap-ci` for deterministic `npm ci` installs, then explicitly runs `npx playwright install chromium` so renderer smoke coverage is exercised on GitHub Actions.
16-
- For v1, CI intentionally follows the major-version tool pins declared in `mise.toml` (`node = "24"`, `python = "3"`). This repo does not commit a `mise.lock` yet.
21+
## Feature highlights
22+
23+
- Full session lifecycle management: create, inspect, list, wait, destroy, and garbage-collect.
24+
- Semantic snapshots for structured or text inspection, including optional scrollback capture.
25+
- Renderer-backed screenshots and replay exports for reviewable visual evidence.
26+
- Recording export to asciicast (`.cast`) or WebM for artifact bundles.
27+
- Failure recovery via reconciliation, stale-session cleanup, and retained manifests/artifacts.
28+
29+
## CLI-wide flags
30+
31+
- `--home <path>`: override the agent-terminal home directory.
32+
- `--timeout-ms <n>`: apply a shared CLI timeout budget in milliseconds.
33+
- `--no-color`: disable ANSI color in human-readable output.
34+
- `--json`: available on user-facing commands to emit structured command envelopes.
35+
36+
## Commands
37+
38+
- `version`: print the CLI version.
39+
- `doctor`: validate local environment requirements.
40+
- `create [command...]`: create a session and launch the requested command or shell.
41+
- `list`: list sessions, optionally including exited ones.
42+
- `inspect <session-id>`: inspect manifest state and artifact metadata for a session.
43+
- `destroy <session-id>`: tear down a session, with optional forced shutdown.
44+
- `gc`: remove stale or old sessions.
45+
- `type <session-id> [text]`: type text into a session.
46+
- `paste <session-id> [text]`: paste text into a session.
47+
- `mark <session-id> <label>`: add a marker event to a session timeline.
48+
- `send-keys <session-id> <keys...>`: send key sequences such as `Enter` or `Ctrl+C`.
49+
- `resize <session-id>`: resize the PTY dimensions.
50+
- `signal <session-id> <signal>`: send a POSIX signal to the session child process.
51+
- `snapshot <session-id>`: capture a semantic snapshot of terminal contents.
52+
- `screenshot <session-id>`: capture a rendered PNG screenshot.
53+
- `record export <session-id>`: export replay artifacts as asciicast or WebM.
54+
- `wait <session-id>`: wait for exit, idleness, text, regex, cursor, or stable-screen conditions.
55+
56+
## Development setup
57+
58+
```bash
59+
mise install
60+
npm ci
61+
npx playwright install chromium
62+
```
63+
64+
Useful shortcuts:
65+
66+
- `mise run bootstrap`: install npm dependencies and Chromium in one step.
67+
- `npm run cli -- --help`: inspect the CLI locally without building.
68+
69+
## Verification
70+
71+
```bash
72+
npm run verify
73+
```
74+
75+
That runs formatting, linting, typechecking, unit/e2e tests, and the production build.
76+
77+
## Design docs
78+
79+
Design and implementation notes live under `design/`, especially `design/20260319_agent-terminal-v1/`.
80+
81+
## Repository notes
82+
83+
- CI uses `mise` for tool provisioning and quality-gate entrypoints.
84+
- Chromium is required locally for screenshot and replay export coverage.
85+
- Dogfood proof bundles and validation notes live under `dogfood/` and `design/`.

WEEK2-GAPS.md

Lines changed: 42 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -1,29 +1,53 @@
1-
# Week 2 remaining gaps
1+
# Remaining gaps after Week 4
22

3-
The Week 2 renderer-backed inspection slice is complete, but the following work is still intentionally out of scope or not yet delivered:
3+
> Historical note: this file keeps its original filename (`WEEK2-GAPS.md`) because earlier design docs and proof bundles already reference it. Its contents now describe the current post-Week-4 delta rather than the original Week 2-only gap list.
44
5-
## Export and packaging
5+
Week 1 control-plane work, Week 2 renderer-backed inspection, Week 3 export / retention, and the core Week 4 CLI / artifact / lifecycle hardening have all landed. The remaining work is now concentrated in the areas below.
66

7-
- **Asciicast export** is not implemented yet.
8-
- **WebM video export** is not implemented yet.
9-
- **MCP wrapper** is not implemented yet.
7+
## Post-Week-4 remaining gaps
108

11-
## Renderer backends and platform coverage
9+
### CLI contract and config parity
1210

13-
- **Native renderer adapters** are not implemented yet; the current slice is centered on the reference `ghostty-web` path.
14-
- **Cross-platform rendering parity** is not guaranteed yet.
11+
- **`--log-level`** is still not implemented.
12+
- **Global render-profile selection** is still partial; `screenshot` exposes a command-local `--profile`, but there is not yet a broader global/profile-default story.
13+
- **`--idle-timeout-ms`** is still not implemented for `create`.
14+
- **`--append-newline`** is still not implemented for `type`.
15+
- **Config-file loading** is still not implemented, and the broader env/config precedence story from `02-cli-contract.md` remains incomplete.
16+
- **Full envelope/result-shape parity** with every CLI-contract example is still incomplete.
1517

16-
## Input and topology
18+
### Artifact fidelity and metadata
1719

18-
- **Mouse input support** is not implemented yet.
19-
- **Remote/network sessions** are not implemented yet.
20+
- **Per-cell style metadata** is still not implemented.
21+
- **The fuller `SnapshotCell` / expanded snapshot schema** from `03-rendering-and-artifacts.md` is still not implemented.
22+
- **Bundled deterministic font assets** are still not implemented; built-in profiles still rely on generic `monospace`.
23+
- **Full replay timing controls** are still not exposed as a complete reviewer-facing CLI surface.
2024

21-
## Fidelity and determinism
25+
### Failure semantics and recovery
2226

23-
- **Screenshot pixel-perfect determinism** is not guaranteed; font rendering can still vary by environment.
24-
- **Scrollback in snapshots** is not implemented; snapshots currently report the visible viewport only.
25-
- **Cursor blink animation in screenshots** is not captured; screenshots represent a static frame.
27+
- **Renderer/host recovery proof** is still lighter than the main event-log/offline-replay story.
28+
- **Broader failure storytelling** is still incomplete; the repo now records `failed` plus `failureReason`, but the docs still sketch richer future distinctions between abnormal child exit, host failure, and renderer failure.
2629

27-
## Security & Isolation
30+
### Fixture suite and dogfooding
2831

29-
- **Renderer CSP trade-off** currently allows `unsafe-inline`/`unsafe-eval` for the ghostty-web harness because the localhost-only loopback renderer still needs inline bootstrap code and WASM eval support in current browsers.
32+
- **Local proof-bundle review helper/page** is still not implemented.
33+
34+
### Platform and future-scope work
35+
36+
- **Native renderer adapters** are still not implemented.
37+
- **Mouse input support** is still not implemented.
38+
- **Remote/network sessions** are still not implemented.
39+
- **MCP wrapper** is still not implemented.
40+
- **Cross-platform rendering parity** is still not guaranteed, and Windows remains behind the design’s intended tier-2 shape.
41+
- **Renderer CSP trade-off** still exists; the localhost-only ghostty-web harness still needs `unsafe-inline` / `unsafe-eval` today.
42+
43+
## Recommended next step
44+
45+
The next milestone should focus on the still-open parity and validation work rather than a brand-new feature family:
46+
47+
1. finish CLI/config parity,
48+
2. finish snapshot/rendering fidelity,
49+
3. finish the remaining validation/tooling work around local proof-bundle review,
50+
4. strengthen failure/recovery validation,
51+
5. then continue broader native/platform future work.
52+
53+
See `design/20260319_agent-terminal-v1/10-week-4-status.md` for the detailed Week 4 status record and `design/20260319_agent-terminal-v1/09-week-4-plan.md` for the original Week 4 plan.

design/20260319_agent-terminal-v1.md

Lines changed: 8 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -19,18 +19,20 @@ It is designed to let an agent:
1919

2020
This design intentionally describes a **general product**, not a Mux-specific implementation. A future Mux integration should consume `agent-terminal` as an external CLI/runtime rather than baking Mux-specific assumptions into the design.
2121

22-
## Current shipped status (2026-03-21)
22+
## Current shipped status (2026-03-22)
2323

24-
The repository now ships the first renderer-backed vertical slice of this design:
24+
The repository now ships the first three milestones of this design:
2525

2626
- long-lived session hosts,
2727
- PTY control and append-only event logs,
2828
- renderer-backed `snapshot` and `wait`,
2929
- deterministic `screenshot`,
30-
- artifact manifests,
30+
- `record export --format asciicast`,
31+
- `record export --format webm`,
32+
- artifact manifests and `gc`,
3133
- and proof bundles under `dogfood/`.
3234

33-
Replay export artifacts such as asciicast and video remain part of the design direction, but they are still future work rather than shipped functionality.
35+
The implementation is now beyond the original Week 2 slice, but it is **not** yet fully done against the broader v1 design. The main remaining gaps are CLI-contract parity, richer snapshot / screenshot fidelity, missing fixture scenarios, and a few failure-state / platform hardening items. See [08-week-3-status.md](./20260319_agent-terminal-v1/08-week-3-status.md), [09-week-4-plan.md](./20260319_agent-terminal-v1/09-week-4-plan.md), and [`../WEEK2-GAPS.md`](../WEEK2-GAPS.md) for the current follow-on work.
3436

3537
## Executive summary
3638

@@ -194,6 +196,8 @@ This design file is the entry point. Detailed supporting docs live in `design/20
194196
- [05-dogfooding-and-validation.md](./20260319_agent-terminal-v1/05-dogfooding-and-validation.md)
195197
- [06-roadmap-and-week-1-plan.md](./20260319_agent-terminal-v1/06-roadmap-and-week-1-plan.md)
196198
- [07-week-2-plan.md](./20260319_agent-terminal-v1/07-week-2-plan.md)
199+
- [08-week-3-status.md](./20260319_agent-terminal-v1/08-week-3-status.md)
200+
- [09-week-4-plan.md](./20260319_agent-terminal-v1/09-week-4-plan.md)
197201

198202
## High-level architecture
199203

design/20260319_agent-terminal-v1/02-cli-contract.md

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -684,3 +684,22 @@ The CLI contract is complete when:
684684
- key grammar is canonicalized and tested,
685685
- `doctor` catches missing browser/render dependencies,
686686
- and the CLI can be driven end-to-end by a non-interactive agent process.
687+
688+
## 24. Week 4 implementation status
689+
690+
As of 2026-03-22, Week 4 closed several of the highest-value CLI contract gaps:
691+
692+
- shipped global root flags `--home`, `--timeout-ms`, and `--no-color` via a shared command context,
693+
- shipped differentiated process exit codes `0` through `8` via structured error-to-exit-code mapping,
694+
- shipped `create` options `--env`, `--term`, `--name`, and `--shell`,
695+
- shipped file-backed input for `type` and `paste` via `--file`,
696+
- and shipped renderer-backed cursor waits via `wait --cursor-row` / `--cursor-col`.
697+
698+
The following contract items remain future work:
699+
700+
- `--log-level`,
701+
- a true global `--profile` override surface,
702+
- `--idle-timeout-ms`,
703+
- `--append-newline`,
704+
- config-file loading and the broader config/env precedence story beyond `AGENT_TERMINAL_HOME`,
705+
- and full JSON envelope/result-shape alignment with every example in this contract.

design/20260319_agent-terminal-v1/03-rendering-and-artifacts.md

Lines changed: 31 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -31,30 +31,33 @@ That distinction must remain explicit in both code and documentation.
3131

3232
V1 should support four artifact classes.
3333

34-
| Artifact | Purpose | Required in v1 |
35-
| ----------------- | ---------------------------------------------------- | -------------- |
36-
| Semantic snapshot | Structured screen state for reasoning and assertions | Yes |
37-
| Screenshot PNG | Visual verification of layout, color, and wrapping | Yes |
38-
| Asciicast | Portable terminal replay artifact | Not yet shipped |
39-
| Replay video | Reviewer-friendly visual playback | Not yet shipped |
34+
| Artifact | Purpose | Required in v1 | Shipped as of 2026-03-22 |
35+
| ----------------- | ---------------------------------------------------- | -------------- | ------------------------ |
36+
| Semantic snapshot | Structured screen state for reasoning and assertions | Yes | Yes |
37+
| Screenshot PNG | Visual verification of layout, color, and wrapping | Yes | Yes |
38+
| Asciicast | Portable terminal replay artifact | Yes | Yes |
39+
| Replay video | Reviewer-friendly visual playback | Yes | Yes |
4040

41-
## Current implementation status (2026-03-21)
41+
## Current implementation status (2026-03-22)
4242

43-
The current Week 2 implementation ships the first two artifact classes from this design:
43+
The current implementation now ships all four artifact classes from this design:
4444

4545
- semantic snapshots,
46-
- and screenshot PNGs.
46+
- screenshot PNGs,
47+
- asciicast export,
48+
- and replay-video export.
4749

48-
It does **not** yet ship asciicast export or replay video export; those remain deferred and are tracked in `WEEK2-GAPS.md`.
49-
50-
The current renderer path is:
50+
The current renderer/export path is:
5151

5252
- host-prepared replay input,
5353
- lazy `ghostty-web` boot in headless Chromium,
5454
- viewport-scoped semantic extraction,
5555
- deterministic screenshot capture,
56+
- deterministic replay export to asciicast and WebM,
5657
- and manifest-backed artifact storage under `artifacts/`.
5758

59+
Remaining follow-on work is now mostly about fidelity and design parity rather than missing artifact classes. Scrollback snapshots and richer screenshot/export metadata now ship; the biggest open items are per-cell style metadata, bundled deterministic font assets, the fuller `SnapshotCell` surface, and more explicit replay timing controls. These are tracked in [`../WEEK2-GAPS.md`](../WEEK2-GAPS.md) and [09-week-4-plan.md](./09-week-4-plan.md).
60+
5861
## 4. Canonical replay model
5962

6063
Everything visual should be reproducible from:
@@ -519,3 +522,19 @@ This area is complete only when:
519522
- replay video export exists and is reviewable,
520523
- every artifact is manifest-backed and hash-stamped,
521524
- and renderer crashes can be repaired from replay without losing the session.
525+
526+
## 18. Week 4 implementation status
527+
528+
As of 2026-03-22, Week 4 materially narrowed the rendering/artifact delta:
529+
530+
- shipped scrollback-aware snapshots through `snapshot --include-scrollback` / `includeScrollback`,
531+
- shipped screenshot metadata enrichment (`rendererBackend`, `pixelWidth`, `pixelHeight`, `sha256`, and `renderProfileHash`),
532+
- shipped render-profile hashing via `hashProfile(...)`,
533+
- and shipped richer asciicast/WebM export metadata for offline review.
534+
535+
The remaining design-level follow-ons are:
536+
537+
- per-cell styling and the fuller `SnapshotCell` contract,
538+
- a bundled deterministic font asset instead of host `monospace`,
539+
- the broader snapshot schema sketched earlier in this document,
540+
- and a fuller replay-timing option surface for reviewer-facing video export.

design/20260319_agent-terminal-v1/05-dogfooding-and-validation.md

Lines changed: 35 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -6,25 +6,51 @@ It is intentionally prescriptive.
66

77
A follow-up AI coding agent should treat this file as the minimum review protocol, not optional guidance.
88

9-
## Current shipped state (2026-03-21)
9+
## Current shipped state (2026-03-22)
1010

11-
This document still describes the *target* dogfooding protocol, but the current shipped product only supports a subset of the artifact expectations below.
11+
This document still describes the broader dogfooding target, but the repository now ships the full core artifact set needed for review bundles.
1212

1313
Shipped today:
1414

1515
- JSON command outputs,
16-
- semantic snapshots,
17-
- PNG screenshots,
16+
- semantic snapshots, including optional scrollback capture,
17+
- PNG screenshots with richer metadata,
18+
- `.cast` export,
19+
- replay video export,
1820
- artifact manifests,
1921
- and notes / proof bundles under `dogfood/`.
2022

21-
Not yet shipped:
23+
The remaining gaps are now mostly about local review ergonomics rather than missing artifact classes. The dedicated `unicode-grid` and `scrollback-demo` fixtures, their matching E2E coverage, and the Week 4 proof bundles are now shipped; the biggest open item is a local bundle review helper.
2224

23-
- `.cast` export,
24-
- replay video export,
25-
- and some of the richer fixture scenarios listed below.
25+
Read the remainder of this file as the broader validation target and checklist for closing those remaining gaps.
26+
27+
## Week 4 coverage
28+
29+
As of 2026-03-22, the repo ships the deterministic `unicode-grid` and `scrollback-demo` fixtures plus dedicated E2E coverage in `test/e2e/unicode-grid.test.ts` and `test/e2e/scrollback-demo.test.ts`.
30+
31+
The repo also now has four Week 4 proof bundles that cover the shipped gap-closing work:
32+
33+
- `dogfood/20260322-week4-cli-parity/` proves CLI-contract parity and result-shape behavior,
34+
- `dogfood/20260322-week4-failure-recovery/` proves failure retention and recovery evidence,
35+
- `dogfood/20260322-week4-scrollback-review/` proves scrollback capture and replay review,
36+
- and `dogfood/20260322-week4-unicode-review/` proves unicode/width review coverage.
37+
38+
Bugfix verification bundles also ship alongside that Week 4 coverage:
39+
40+
- `dogfood/20260323-bugfix-resize/` proves post-resize snapshot/screenshot recovery,
41+
- and `dogfood/20260323-bugfix-scrollback/` proves scrollback viewport normalization.
42+
43+
Earlier proof bundles remain relevant context:
44+
45+
- `dogfood/20260321-post-hardening-smoke/` revalidates live inspect/wait/snapshot/screenshot/doctor flows,
46+
- `dogfood/20260321-week3-renderer-complete/` proves live and post-exit snapshot/screenshot/export plus GC,
47+
- `dogfood/20260321-week3-crash-retention/` proves abnormal-exit evidence retention,
48+
- `dogfood/20260322-global-cli-context/` proves `--home`, `--no-color`, and exit-code differentiation for missing sessions,
49+
- and `dogfood/20260322-lazyvim-scenario/` demonstrates real-world TUI driving plus screenshot / asciicast / WebM review artifacts.
50+
51+
The main remaining validation gap is:
2652

27-
Read the remainder of this file as the broader validation target, not a claim that every artifact class is already implemented.
53+
- there is still no local proof-bundle review helper/page.
2854

2955
## 1. Dogfooding goals
3056

0 commit comments

Comments
 (0)