Skip to content

Close projection iterators before handling or checkpointing - #57

Merged
jefflinse merged 2 commits into
mainfrom
compare/opencode-processor
Sep 21, 2026
Merged

jefflinse merged 2 commits into
mainfrom
compare/opencode-processor

Conversation

@jefflinse

Copy link
Copy Markdown
Contributor

The processor currently handles events and saves checkpoints while its ReadAll iterator remains open. A reader and checkpoint store backed by the same bounded connection pool can therefore deadlock while each processor holds one connection and waits for another.

This change reads events into bounded batches, closes each iterator successfully, and only then invokes handlers and checkpoint saves. The default batch size is 500, and non-positive explicit batch sizes are rejected so an unbounded history is never buffered in memory. Close failures are fatal and suppress every callback from that batch; accepted prefixes still run before later read failures, with handler and checkpoint failures retaining their existing precedence.

Cancellation is checked throughout collection, after close, between buffered events, and around the durable head checkpoint. A full batch triggers another immediate read and only a short read can establish the store head. Iterator and fold documentation now makes resource ownership explicit.

Verification:

  • go test -count=1 -race ./...
  • go vet ./...
  • go build ./...
  • golangci-lint run

Closes #54

@jefflinse
jefflinse merged commit dac3ae3 into main Sep 21, 2026
6 checks passed
@jefflinse
jefflinse deleted the compare/opencode-processor branch September 21, 2026 21:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Processor holds its read iterator across saveCheckpoint, requiring two pool connections per processor

1 participant