Skip to content

fix(storage): prune redundant intermediate message_update frames - #1279

Merged
wyuc merged 3 commits into
mainfrom
fix/prune-message-update-frames
Aug 29, 2026
Merged

fix(storage): prune redundant intermediate message_update frames#1279
wyuc merged 3 commits into
mainfrom
fix/prune-message-update-frames

Conversation

@wyuc

@wyuc wyuc commented Aug 29, 2026

Copy link
Copy Markdown
Contributor

Fixes #1278.

What

After a message_end is durably appended, prune the intermediate message_update rows of that message's contiguous update run, keeping only the first and last. This materializes at write-time exactly what the read path already discards.

Why

pi streams an assistant message as many message_update events and every frame is persisted carrying the full accumulated message. The read path (readEventsAfterForReplay / client compactReplayEvents) already keeps only the first+last of each contiguous run, message_end carries the full text, and crash/resume never reads message_update (it reconstructs from the entry tree + pi transcript). So intermediate frames are persisted but never consumed — pure write amplification (measured ~74% of rows / ~89% of bytes in one deployment).

How

  • New PgAgentSessionStore.pruneMessageUpdates(sessionId, messageEndSeq): walks backward from the persisted message_end via the (session_id, seq) primary-key index, bounds the contiguous message_update run immediately preceding it, and deletes only rows strictly between the run's first and last frames. O(run length), not O(session); never touches rows at/after messageEndSeq (in-flight next message).
  • Runner calls it after message_end is durably appended, ordered in the write chain after that append, off the live-stream path; prune failures are logged and do not fail the run.

Safety

  • No schema change; read path untouched.
  • Replay output verified byte-equivalent before/after pruning.
  • In-flight (not-yet-ended) frames untouched; single-frame/empty runs are no-ops.
  • Crash/resume unaffected (doesn't use message_update).

Tests

  • Physical first+last retention + deletion count; idempotent second call.
  • Preservation of message boundaries, unrelated events, and next-message frames.
  • Replay equivalence before/after pruning.
  • Runner invocation with the committed message_end seq and correct ordering.
  • Existing agent-session contract suite: no assertions changed, all pass.

Validation

tsc --noEmit, storage schema suites (66 passed / 37 env-gated skipped), runner suites (9 passed), ESLint, repo-wide pnpm check.

@wyuc
wyuc marked this pull request as ready for review August 29, 2026 17:31
@wyuc
wyuc merged commit f8b30ef into main Aug 29, 2026
5 checks passed
@wyuc
wyuc deleted the fix/prune-message-update-frames branch August 29, 2026 17:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

agent_session_events write amplification: redundant intermediate message_update frames

1 participant