Add active-turn memory pressure and browser-session admission controls

[hamzamerzic]

Why

PR #129 bounds leaked process/browser lifetime at terminal turn cleanup, but it deliberately does not kill resources while a valid turn is still running. A long-running turn can still accumulate several explicitly named agent-browser sessions or leave a memory-heavy tool child alive after the command item reports completion.

Observed on one 6 GiB production cgroup:

• memory.events: oom=3, oom_kill=1, with thousands of max hits
• concurrent Chromium trees were the dominant live footprint
• multiple custom browser sessions were created by one chat while its inherited per-chat session already existed
• a naive whole-binary Python scan continued CPU-heavy after the parent shell/tool wrapper ended

No chat content or credentials are needed to reproduce or monitor this class.

Needed invariants

• Prefer/reuse the inherited AGENT_BROWSER_SESSION; make an explicit second session opt-in and bounded per chat.
• Expose cgroup memory.current, memory.max, memory.events, browser-session count, and owned descendant count on the operator health surface.
• Before starting discretionary heavy work (new browser tree, app build, additional agent turn), apply a pressure gate with hysteresis rather than waiting for kernel OOM.
• Attribute every live browser daemon and command descendant to a chat/turn; terminal/Stop cleanup must be idempotent and identity-keyed.
• Decide an active-turn policy for a command that has reported completion while descendants remain (kill by default, or require an explicit background-process lease with TTL).
• Do not kill a successor turn's resources or shared platform services.

Suggested verification

• adversarial command spawns a TERM-resistant child, exits its shell, and proves the child is reclaimed at the chosen boundary
• one chat creates multiple custom browser sessions; the configured cap/reuse path prevents unbounded Chrome trees
• two concurrent chats prove cleanup/admission is attribution-safe
• pressure-gate tests cover enter/high-water, remain-gated, low-water recovery, and unavailable-cgroup fallback
• Stop, graceful restart, OOM recovery, and normal terminal completion converge to no owned descendants

Related: #129.

[hamzamerzic]

Fresh post-deploy production evidence (2026-07-22 19:46–19:50 UTC) reinforces this issue after #129 containment landed:

• One valid resolver turn opened a single chat-scoped agent-browser profile for a final UI check.
• Two browser commands (wait --text ..., followed by snapshot) remained reported as running after the source merge, package tests, and bundle promotion had already completed.
• The turn's process tree grew the main container from 40 to 284 PIDs. Its fresh cgroup memory.events.max rose from 24 to 217 and memory.peak reached the 6 GiB ceiling; oom=0 and oom_kill=0.
• A normal authenticated Stop transitioned the durable run to stopped, cleared the chat marker, and—through Contain Codex descendants and custom browser sessions #129—reclaimed the Codex, agent-browser, and Chrome descendants. PIDs returned to 40 immediately; no manual signal was sent to a child.

This is the exact remaining boundary described here: terminal/Stop cleanup now converges, but an otherwise-complete active turn can still retain a large browser tree and hit the limit while waiting on a wedged tool. Admission/hysteresis, per-turn browser caps/reuse, and a TTL/lease for tool descendants would have prevented the pressure episode before Stop became necessary.

[hamzamerzic]

Two more independent post-deploy reproductions occurred during the final Atlas and News package reconciliations (2026-07-22 20:05–20:10 UTC):

• Both app updates had already finalized through the normal watcher, and their complete source/package tests had passed.
• Each turn then wedged on a final agent-browser wait --text ... against content that never appeared. Atlas reached 292 PIDs and 6,025,089,024 bytes; News reached 294 PIDs and 6,082,543,616 bytes in the 6 GiB app cgroup.
• The cgroup remained at its prior memory.events.max=217, with oom=0 and oom_kill=0, so neither episode crossed the limit. Anonymous memory after cleanup was about 857 MiB; most of the remaining memory.current was reclaimable file cache from package/test activity.
• In both cases a normal authenticated Stop durably marked the run stopped and reclaimed the entire chat-scoped browser/Chrome tree. Final production state returned to 41 PIDs with no agent-browser, Chrome, Codex, Claude, or test descendants.

This repeated twice in different apps within minutes and confirms the same split: #129 cleanup/containment converges correctly once Stop runs, while a valid active turn still lacks a bounded wait/lease, per-turn browser budget, and pre-limit admission control.

[hamzamerzic]

New independent evidence from a Railway Trial deployment shows this failure class at the much tighter 1 GiB boundary:

• cgroup limit was ~953 MiB; a quiet reading was ~820 MiB (~86%)
• memory.events had recorded 8 ceiling hits since a restart only ~2 hours earlier
• chat 428c0ab5… logged stop_chat_for: handle.stop() timed out at about 07:13
• the stalled-live watchdog retried the graceful interrupt roughly every 10 minutes through 09:15 and timed out each time
• a Railway restart at 09:21 came back cleanly and startup reconciled exactly 1 interrupted chat

This exposes a concrete gap beneath the broader admission work here: both Stop and the watchdog reused the same graceful SDK interrupt indefinitely. There was no identity-scoped hard-stop transition, and the outer asyncio.wait_for() could cancel the handle's shared _finished future while timing out.

PR #159 implements that bounded slice:

• keeps graceful interrupt first
• preserves the completion future across the 2s bound
• isolates Claude turns into private process groups (Codex already had this)
• rechecks exact registry-handle identity before a one-shot TERM/KILL escalation
• never signals a retained PGID twice
• closes the chat-attributed detached agent-browser sessions after escalation
• leaves an unconfirmed runner registered for existing reconciliation

It does not close the remaining #130 work: cgroup observability, pressure/admission hysteresis, active-turn/tool leases, and per-turn browser budgets are still needed to prevent a valid-but-heavy turn from reaching the limit in the first place.