feat: process session events as durable causal runs

docs/architecture.md

@@ -29,10 +29,22 @@ flowchart LR
 
 ### The server owns history
 
-The event log is the source of truth for a session. Before each model turn the
-application reads committed events and projects them into a Messages API
-conversation. The model endpoint performs inference; it does not own session
-state.
+The event log is the source of truth for a session, but two different orderings
+are read from it:
+
+- **Public event history** (`GET .../events`, list, and the live SSE stream) is
+  the immutable receipt/commit sequence. It never reorders or hides events.
+- **Model-facing conversation order** is reconstructed per turn from run
+  causality, not from raw commit order. For each prior completed run, in
+  admission order, the projection replays that run's trigger event IDs followed
+  by its persisted output event IDs, then appends the current run's trigger. The
+  output association is durable state on the run, so this ordering is rebuilt
+  identically after a restart, and a run never sees a later trigger that was
+  queued while it was still running.
+
+Before each model turn the application reconstructs that causal history and
+projects it into a Messages API conversation. The model endpoint performs
+inference; it does not own session state.
 
 ### Wire and domain models are separate
 
@@ -75,9 +87,9 @@ or sandbox dependencies.
 ## Durable write path
 
 Submitting input is not “write an event, then enqueue work.” The store commits
-the client events, `session.status_running`, session projection, and queued run
-in one transaction. A crash therefore cannot leave accepted input without
-corresponding work.
+the client events, `session.status_running`, session projection, and one queued
+run per processable trigger (in admission order) in one transaction. A crash
+therefore cannot leave accepted input without corresponding work.
 
 Runtime calls happen outside SQL transactions. At run completion, buffered
 authoritative output, trigger `processed_at`, the final session status, and run
@@ -108,20 +120,21 @@ ownership and duplicate-side-effect risks.
 
 The strongest current risks are semantic rather than structural:
 
-1. A batch of multiple triggering input events is admitted as one run, while
-   each trigger currently reprojects only committed history. Output buffered by
-   an earlier trigger in the same claim is not visible to the next trigger.
-2. Runtime output is buffered until completion even though tools may already
+1. Runtime output is buffered until completion even though tools may already
    have performed side effects. A process crash can therefore repeat a side
    effect without a durable journal of the prior attempt.
-3. Pending client actions are encoded in events and stop reasons rather than a
-   first-class durable `pending_actions` model.
-4. Sandboxes are session-scoped: a session's logical sandbox is provisioned on
+2. Pending client actions are encoded in events and stop reasons rather than a
+   first-class durable `pending_actions` model. A single custom-tool park and
+   its `user.custom_tool_result` resume are implemented and tested, but there is
+   no durable pending-action gate: if other trigger runs were already queued
+   before a run parks with `requires_action`, their gating and correlation
+   against the parked action are not yet fully modeled.
+3. Sandboxes are session-scoped: a session's logical sandbox is provisioned on
    first tool use, reused across its runs, and released on session deletion.
    The manager is in-memory, so a process restart does not restore an idle
    session's workspace, and there is no durable checkpoint, quota, or eviction
    policy yet.
-5. `SessionService` currently combines session CRUD, admission, dispatch, and
+4. `SessionService` currently combines session CRUD, admission, dispatch, and
    completion orchestration. These responsibilities should be separated before
    introducing multiple workers or richer retry behavior.
 

docs/architecture/session-lifecycle.md

@@ -31,12 +31,12 @@ sequenceDiagram
   participant M as Messages API
   participant S as Sandbox
 
-  C->>A: POST user.message
-  A->>DB: input + status_running + queued run
+  C->>A: POST user.message (batch)
+  A->>DB: input + status_running + one queued run per trigger
   DB-->>A: commit
   A-->>C: accepted input events
-  A->>DB: claim oldest run for session
-  A->>DB: read committed event history
+  A->>DB: claim next queued run (admission order)
+  A->>DB: reconstruct causal run history
   A->>R: Run(snapshot, projected messages)
   R->>M: create streamed message
   M-->>R: text and/or tool_use
@@ -48,20 +48,51 @@ sequenceDiagram
   R-->>A: buffered authoritative events
   A->>DB: output + processed_at + final status + completion
   DB-->>A: commit
+  Note over A,DB: next run is claimed only after this commit,<br/>so it projects the previous run's committed output
 ```
 
 ## Admission invariant
 
-The server commits all of the following atomically:
+The server commits all of the following atomically, in submitted input order:
 
 - submitted client events;
 - a `session.status_running` event when the session was not already running;
 - the mutable session status;
-- one queued internal run referencing the submitted event IDs.
+- one queued internal run **per processable trigger event**, in admission
+  order. A run references exactly one trigger; multiple triggers are never
+  grouped into a single run.
 
 The client is never told that input was accepted unless the corresponding work
 item is durable.
 
+## Two orderings of the event log
+
+The event log is read in two distinct orders:
+
+- **Public event history** — `GET .../events`, list, and the live SSE stream —
+  is the immutable receipt/commit sequence. It is never reordered.
+- **Model-facing conversation order** is reconstructed per turn from run
+  causality. For each prior completed (or failed) run, in admission order, the
+  projection replays that run's trigger event IDs followed by its persisted
+  output event IDs, then appends the current run's trigger. The run's output
+  event IDs are durable state committed in the same transaction that closes the
+  run, so this ordering survives a restart and a run never sees a later trigger
+  queued while it was still running. Tested by
+  `TestRunStore_ModelHistorySurvivesReopenInCausalOrder` (file-backed reopen)
+  and `TestSessionService_BatchedTriplePerRunCausalProjection` (three batched
+  triggers project as three causal turns).
+
+## Per-run boundary
+
+Runs drain one at a time in admission order. A run commits its buffered
+authoritative output and stamps its trigger processed *before* the next run is
+claimed, so each run's causal history already includes the previous run's
+committed output — a later user event in the same batch sees the earlier agent
+reply (`TestRunStore_CompletionBeforeNextClaimObservesOutput`,
+`TestSessionService_SecondUserEventObservesFirstAgentOutput`). A terminated
+session is final: its leftover queued runs are never claimed, and it is never
+flipped back to `running`.
+
 ## Per-session ordering
 
 A partial unique database index permits at most one `running` item per session.
@@ -102,5 +133,11 @@ A custom tool or `always_ask` built-in can park a run:
 3. the stop reason names the event IDs the client must answer;
 4. a custom tool response starts a new durable run.
 
-Custom-tool resume is implemented. Built-in `user.tool_confirmation` is
-accepted by the HTTP API, but its allow/deny resume semantics are not complete.
+Custom-tool resume is implemented and tested for a single park/result cycle.
+Built-in `user.tool_confirmation` is accepted by the HTTP API, but its
+allow/deny resume semantics are not complete.
+
+There is no first-class durable pending-action gate. If other trigger runs were
+already queued before a run parks with `requires_action`, their gating and
+correlation against the parked action are not yet fully modeled; this case is
+not claimed to work.

docs/compatibility.md

@@ -52,7 +52,7 @@ tests in `internal/httpapi/sdk_golden_test.go`.
 |---|---|---|---|
 | Event is a flat top-level tagged union (`{id,type,...,processed_at}`) | partial | `TestGolden_EventIsFlatTaggedUnion` | The raw golden covers one `user.message`; the full event union and exact per-variant keys are not covered. |
 | `user.message` with `content[]` blocks | partial | `TestSDK_EventSendAndList`, `TestGolden_EventIsFlatTaggedUnion` | One text block round-trips; other block variants and validation are incomplete. |
-| Send events (`POST .../events`) echoes submitted events | partial | `TestSDK_EventSendAndList`, `TestSessionService_InitialEventBatchProcessesEveryEventInOrder`, `TestSessionService_SendEventBatchProcessesEveryEventInOrder`, `TestRunStore_QueuesRunsPerSessionInAdmissionOrder` | Same-request and cross-request batches enter a durable per-session order and interrupted runs are requeued on restart. All event variants, cancellation, and external-side-effect idempotency remain incomplete. |
+| Send events (`POST .../events`) echoes submitted events | partial | `TestSDK_EventSendAndList`, `TestSessionService_InitialEventBatchProcessesEveryEventInOrder`, `TestSessionService_SendEventBatchProcessesEveryEventInOrder`, `TestRunStore_QueuesRunsPerSessionInAdmissionOrder`, `TestRunStore_AdmitBatchCreatesOneRunPerTrigger`, `TestRunStore_CompletionBeforeNextClaimObservesOutput`, `TestSessionService_SecondUserEventObservesFirstAgentOutput`, `TestSessionService_BatchedTriplePerRunCausalProjection`, `TestRunStore_ModelHistorySurvivesReopenInCausalOrder` | A batch is admitted atomically in input order; each processable trigger gets its own durable queued run (never grouped), and runs drain one at a time. Model-facing history is reconstructed from run causality (each prior run's trigger IDs then its persisted output IDs, then the current trigger), so a later trigger's projection observes the earlier run's committed agent output as a separate turn — proven for A/B (`TestSessionService_SecondUserEventObservesFirstAgentOutput`) and A/B/C (`TestSessionService_BatchedTriplePerRunCausalProjection`), and durable across a file-backed reopen (`TestRunStore_ModelHistorySurvivesReopenInCausalOrder`). All event variants, interrupt cancellation, and external-side-effect idempotency remain incomplete. |
 | Reject unknown / server-only event types and validate client variants | partial | `TestGolden_RejectsServerOnlyEventType`, `TestSendEvents_ValidatesVariantShape` | Type gating, empty-batch rejection, and validation for the currently modeled client variants are exercised; the full client-event union and every field constraint are not. |
 | List events (`GET .../events`): `types` filter, `limit`, `page`, `next_page` | partial | `TestSDK_EventListPaginationAndTypesFilter`, `TestListEvents_CursorIsBoundToSessionAndFilters` | Cursors are bound to the session and normalized filters. Bounds and `processed_at` ordering/keyset semantics remain incomplete. |
 | List events: `order`, `created_at[gt|gte|lt|lte]` filters | partial | `TestQuery_CreatedAtNamedFilterUsesProcessedAt`, `TestQuery_ProcessedAtFiltersExactAndFractionalWithinSecond`, `TestListEvents_RejectsInvalidQueryValues` | The public `created_at` query name compares a fixed-width `processed_at` key. Results and cursors are still ordered by internal sequence rather than a `processed_at` keyset, including its null/tie semantics. |
@@ -74,7 +74,7 @@ tests in `internal/httpapi/sdk_golden_test.go`.
 | Custom-tool handoff (`agent.custom_tool_use` → park → `user.custom_tool_result` → resume) | partial | `TestWorkshop_CustomToolHandoff`, `TestSessionService_CustomToolParksAndResumes`, `TestAgentCore_CustomToolParksWithRequiresAction`, `TestFake_CustomToolResultEndsIdle`, `TestProjectMessages_CustomToolResultPairing` | A custom tool call parks the run at `session.status_idle` with `stop_reason.type == "requires_action"` and `event_ids` naming the committed `agent.custom_tool_use`; a `user.custom_tool_result` referencing that id resumes a fresh run to `end_turn`. End-to-end proven; the exact `requires_action` payload shape is unconfirmed against the official wire. |
 | Built-in tool run end-to-end via the app layer | partial | `TestSessionService_BuiltinToolRunEndToEnd` | A session-driven run provisions a sandbox, executes a built-in tool, and reaches `end_turn` through the durable run path. |
 | `always_ask` permission policy on built-in tools | partial | `TestAgentCore_AlwaysAskBuiltinParks` | An `always_ask` built-in call parks the run (`agent.tool_use{evaluated_permission:"ask"}` + `requires_action`); the **resume** path (`user.tool_confirmation` → projected `tool_result` + built-in execution) is not wired. `ProjectMessages` drops the dangling `tool_use` so the parked call never poisons a real request (`TestProjectMessages_DropsDanglingToolUse`, `TestProjectMessages_DropsDanglingCustomToolUse`). |
-| Session-scoped sandbox lifecycle (`internal/sandbox` `SessionManager`) | partial | `TestSessionManager_ReusesSandboxPerSession`, `TestSessionManager_IsolatesSessions`, `TestSessionManager_ReleaseDestroysExactlyOnce`, `TestSessionService_SandboxPersistsAcrossRuns`, `TestSessionService_SandboxIsolatedBetweenSessions`, `TestSessionService_SandboxProvisionedOncePerSession`, `TestSessionService_IdleDoesNotDestroySandbox`, `TestSessionService_DeleteReleasesSandboxExactlyOnce` | A sandbox is **scoped to the session**, not the run. The first run needing tools provisions one logical sandbox; later runs in the same session reuse it, so tool-produced file state **persists across turns**. Different sessions get distinct sandboxes and stay isolated. Entering idle does not destroy the sandbox; deleting the session releases it exactly once. Ownership lives in a session-scoped manager that wraps the provider inside the `sandbox` package; `AgentRuntime` receives a resolved sandbox and is unaware of the lifecycle. The manager is in-memory: a process restart does not restore an idle session's sandbox, and there is no durable checkpoint, quota, or eviction yet. |
+| Session-scoped sandbox lifecycle (`internal/sandbox` `SessionManager`) | partial | `TestSessionManager_ReusesSandboxPerSession`, `TestSessionManager_IsolatesSessions`, `TestSessionManager_ReleaseDestroysExactlyOnce`, `TestSessionService_SandboxPersistsAcrossRuns`, `TestSessionService_SandboxIsolatedBetweenSessions`, `TestSessionService_SandboxProvisionedOncePerSession`, `TestSessionService_IdleDoesNotDestroySandbox`, `TestSessionService_DeleteReleasesSandboxExactlyOnce`, `TestSessionService_DeleteTeardownSurvivesRequestCancellation` | A sandbox is **scoped to the session**, not the run. The first run needing tools provisions one logical sandbox; later runs in the same session reuse it, so tool-produced file state **persists across turns**. Different sessions get distinct sandboxes and stay isolated. Entering idle does not destroy the sandbox; deleting the session releases it exactly once. Delete runs that teardown *after* the durable delete has committed, on a context detached from the request context (`context.WithoutCancel`) and bounded by its own timeout, so a client disconnect cannot cancel an in-flight `Destroy` yet a stuck provider cannot hang forever. Ownership lives in a session-scoped manager that wraps the provider inside the `sandbox` package; `AgentRuntime` receives a resolved sandbox and is unaware of the lifecycle. The manager is in-memory: a process restart does not restore an idle session's sandbox, and there is no durable checkpoint, quota, or eviction yet. |
 | Local sandbox (`Provider`/`Sandbox`, restricted local process) | partial | `TestLocal_ExecEcho`, `TestLocal_FileRoundTripAndConfinement`, `TestLocal_Timeout` | `internal/sandbox` provides a two-layer interface and a local-process default that confines paths to a work dir, clears the environment, applies a timeout, and caps output. **Dev-grade guardrail, not a security boundary — do not run untrusted code.** Sandboxes are session-scoped (see the row above): provisioned on first tool use and reused across the session's runs. |
 | Docker sandbox (real-isolation `Provider`, opt-in) | partial | `TestDocker_*` (skipped without a daemon), `TestResolveSandboxProvider_DefaultsToLocal` | The same `Provider`/`Sandbox` interface has a Docker-backed implementation (shells out to the `docker` CLI, no extra module dependency). It gives **real isolation**: each sandbox is a container with its own Linux namespaces/cgroups, a separate filesystem, and `--network none` by default. Selected at startup via `MANAGED_AGENT_SANDBOX=docker` (default is local); image via `MANAGED_AGENT_SANDBOX_IMAGE` (defaults `alpine:latest`). gVisor (`--runtime=runsc`) can layer under the same interface later with no interface change. Not audited for hostile multi-tenant use (shared host kernel). Docker tests are gated on a running daemon and skip in default offline CI. |
 | MCP toolsets | unsupported | `TestParseTools_BuiltinCustomMCP` | Parsed into `domain.ToolSet` but never resolved or executed. |
@@ -105,16 +105,18 @@ tests in `internal/httpapi/sdk_golden_test.go`.
 
 - `domain.ProjectMessages` merges adjacent same-role events into one message
   (concatenating their content blocks in order) so the projected conversation
-  always alternates roles and forms a legal Messages-API request. This covers
-  the two real flows that produce consecutive same-role events — several
-  `user.message` events arriving before `drainRuns` claims them, and a model
-  turn that emits no text (no `agent.message`) leaving two user turns adjacent
-  (`TestProjectMessages_MergesConsecutiveUsers`,
-  `TestProjectMessages_MergesConsecutiveAssistantsAndAlternates`). Within a
-  single `drainRuns` claim every trigger reuses one projected snapshot: reuse is
-  legal (the snapshot alternates), but the snapshot can be stale relative to
-  drafts produced by an earlier trigger in the same claim. Defining and fixing
-  this batch-turn boundary is the first roadmap item.
+  always alternates roles and forms a legal Messages-API request. Under the
+  causal Run history model, multiple `user.message` events queued before
+  `drainRuns` claims them are **not** collapsed into one projected user message:
+  each processable trigger runs as its own durable run and projects as a
+  separate turn (`TestSessionService_SecondUserEventObservesFirstAgentOutput`,
+  `TestSessionService_BatchedTriplePerRunCausalProjection`), and the causal
+  ordering survives a restart (`TestRunStore_ModelHistorySurvivesReopenInCausalOrder`).
+  Merging still fires only when a turn genuinely emits no assistant content —
+  e.g. a model turn that produces no `agent.message`, leaving two user turns
+  adjacent (`TestProjectMessages_MergesConsecutiveUsers`,
+  `TestProjectMessages_MergesConsecutiveAssistantsAndAlternates`). Context
+  compaction when a session exceeds the projection limit is still deferred.
 - Server shutdown does not cancel in-flight model calls. `drainRuns` runs on
   `context.Background()`, so a SIGTERM does not propagate cancellation to an
   in-progress Messages-API request; cancellation propagation is not implemented.
@@ -144,6 +146,12 @@ tests in `internal/httpapi/sdk_golden_test.go`.
   established.
 - Opaque `multiagent` input is stored with tested replace/null-clear behavior;
   roster resolution, reference validation, and multiagent execution are absent.
+- There is no first-class durable pending-action gate. A single custom-tool
+  park and its `user.custom_tool_result` resume are implemented and tested
+  (`TestSessionService_CustomToolParksAndResumes`), but if other trigger runs
+  were already queued before a run parks with `requires_action`, their gating
+  and correlation against the parked action are not yet fully modeled; this case
+  is not claimed to work.
 - The durable queue is single-process and restart recovery is at-least-once. A
   crash after an external side effect but before completion commit may replay
   it; no lease, fencing, or idempotency-key protocol exists yet.