Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
53 changes: 0 additions & 53 deletions .changeset/correctness-fixes-0-8-0.md

This file was deleted.

15 changes: 0 additions & 15 deletions .changeset/share-groups-v2.md

This file was deleted.

55 changes: 55 additions & 0 deletions packages/client/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,60 @@
# @kafkats/client

## 0.8.0

### Minor Changes

- fe140c0: ShareConsumer: Kafka 4.2 share groups GA support (ShareFetch/ShareAcknowledge v2)
- KIP-1206: new `acquireMode` config (`'batch_optimized'` default | `'record_limit'`) — `record_limit` strictly caps each fetch at `maxRecords`.
- KIP-1222: new `message.renew()` to extend the acquisition lock without finalizing delivery; safe to call multiple times per message.
- `ShareAcknowledgeResponse.acquisitionLockTimeoutMs` is now decoded on v2 responses.
- AckManager dedupes same-offset RENEW + finalizing-ack pairs and collapses duplicate same-offset RENEWs so the wire never carries overlapping batches.
- `@experimental` markers removed; share groups are GA in Kafka 4.2.
- Backward compatible: against Kafka 4.1 brokers the client negotiates v1 and produces byte-identical wire output to before this change. `record_limit` and `renew()` throw with a clear "requires v2 (Kafka 4.2+)" error against older brokers.

### Patch Changes

- ca7d22f: Correctness and reliability fixes across the producer, consumer, client protocol, and flow state stores.

### Producer
- `flush()` now waits for deferred drains so it no longer resolves while writes are still buffered.
- `transactionalSend` freezes its partition set up front to avoid double-calling the partitioner.
- Retry `TxnOffsetCommit` on retriable and rebalance-in-progress errors instead of failing the transaction.
- Fence the producer on `OUT_OF_ORDER_SEQUENCE_NUMBER` to surface the idempotence violation rather than silently corrupting the stream.
- Reject orphaned in-flight send promises during disconnect so callers are not left hanging.

### Consumer
- Always reschedule the heartbeat after non-fatal errors, preventing the consumer from dropping out of the group.
- Surface auto-commit errors to the consumer instead of swallowing them.
- Clear only the successfully committed partitions on a partial commit failure so the rest are retried.
- Fence stale fetch responses across partition replacement to avoid delivering records from a revoked assignment.

### Client / protocol
- Guard against varint shift overflow in the decoder.
- Close a `ConnectionPool.acquire` race that could hand out a connection mid-teardown.
- Clamp the SASL reauthentication delay to the `setTimeout` maximum.
- Don't await a response on `Produce` with `acks=0` (the broker sends none).
- Always use the v0 response header for `ApiVersions`, matching broker behavior across versions.

### Share consumer
- Release (rather than acknowledge) the last yielded message on shutdown so unprocessed records are redelivered.

### Flow
- Write to the local store before the changelog (local-first ordering) to shrink the inconsistency window on crash.
- Serialize the EOS commit triggered by a rebalance against the next message, and close the EOS rebalance offset-commit gap via an awaitable rebalance hook (shared with the consumer fix above).
- Use stream time (not wall-clock time) for window expiry.
- Implement hopping windows (`TimeWindows.advanceBy`).
- Throw on `SlidingWindows` instead of silently misbehaving (not yet implemented).
- Correct `TableGroupByNode` mapping ordering.
- Apply retention to `reduce` and session aggregators, emitting changelog tombstones for expired state.
- Surface checkpoint persistence errors instead of swallowing them.

### flow-state-lmdb
- Make `range()` upper bound inclusive, matching the in-memory store.
- Await `fsync` on checkpoint writes so a checkpoint is durable before it is reported complete.
- Correct `WindowStore` time-bound math.
- Use order-preserving signed-i64 encoding for time keys so range scans iterate in correct chronological order.

## 0.7.0

### Minor Changes
Expand Down
2 changes: 1 addition & 1 deletion packages/client/package.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"name": "@kafkats/client",
"private": false,
"version": "0.7.0",
"version": "0.8.0",
"description": "Pure-protocol Kafka client, consumer/producer, and EOS patterns in TypeScript.",
"type": "module",
"main": "dist/index.js",
Expand Down
59 changes: 59 additions & 0 deletions packages/codec-zod/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,64 @@
# @kafkats/codec-zod

## 0.8.0

### Minor Changes

- fe140c0: ShareConsumer: Kafka 4.2 share groups GA support (ShareFetch/ShareAcknowledge v2)
- KIP-1206: new `acquireMode` config (`'batch_optimized'` default | `'record_limit'`) — `record_limit` strictly caps each fetch at `maxRecords`.
- KIP-1222: new `message.renew()` to extend the acquisition lock without finalizing delivery; safe to call multiple times per message.
- `ShareAcknowledgeResponse.acquisitionLockTimeoutMs` is now decoded on v2 responses.
- AckManager dedupes same-offset RENEW + finalizing-ack pairs and collapses duplicate same-offset RENEWs so the wire never carries overlapping batches.
- `@experimental` markers removed; share groups are GA in Kafka 4.2.
- Backward compatible: against Kafka 4.1 brokers the client negotiates v1 and produces byte-identical wire output to before this change. `record_limit` and `renew()` throw with a clear "requires v2 (Kafka 4.2+)" error against older brokers.

### Patch Changes

- ca7d22f: Correctness and reliability fixes across the producer, consumer, client protocol, and flow state stores.

### Producer
- `flush()` now waits for deferred drains so it no longer resolves while writes are still buffered.
- `transactionalSend` freezes its partition set up front to avoid double-calling the partitioner.
- Retry `TxnOffsetCommit` on retriable and rebalance-in-progress errors instead of failing the transaction.
- Fence the producer on `OUT_OF_ORDER_SEQUENCE_NUMBER` to surface the idempotence violation rather than silently corrupting the stream.
- Reject orphaned in-flight send promises during disconnect so callers are not left hanging.

### Consumer
- Always reschedule the heartbeat after non-fatal errors, preventing the consumer from dropping out of the group.
- Surface auto-commit errors to the consumer instead of swallowing them.
- Clear only the successfully committed partitions on a partial commit failure so the rest are retried.
- Fence stale fetch responses across partition replacement to avoid delivering records from a revoked assignment.

### Client / protocol
- Guard against varint shift overflow in the decoder.
- Close a `ConnectionPool.acquire` race that could hand out a connection mid-teardown.
- Clamp the SASL reauthentication delay to the `setTimeout` maximum.
- Don't await a response on `Produce` with `acks=0` (the broker sends none).
- Always use the v0 response header for `ApiVersions`, matching broker behavior across versions.

### Share consumer
- Release (rather than acknowledge) the last yielded message on shutdown so unprocessed records are redelivered.

### Flow
- Write to the local store before the changelog (local-first ordering) to shrink the inconsistency window on crash.
- Serialize the EOS commit triggered by a rebalance against the next message, and close the EOS rebalance offset-commit gap via an awaitable rebalance hook (shared with the consumer fix above).
- Use stream time (not wall-clock time) for window expiry.
- Implement hopping windows (`TimeWindows.advanceBy`).
- Throw on `SlidingWindows` instead of silently misbehaving (not yet implemented).
- Correct `TableGroupByNode` mapping ordering.
- Apply retention to `reduce` and session aggregators, emitting changelog tombstones for expired state.
- Surface checkpoint persistence errors instead of swallowing them.

### flow-state-lmdb
- Make `range()` upper bound inclusive, matching the in-memory store.
- Await `fsync` on checkpoint writes so a checkpoint is durable before it is reported complete.
- Correct `WindowStore` time-bound math.
- Use order-preserving signed-i64 encoding for time keys so range scans iterate in correct chronological order.

- Updated dependencies [ca7d22f]
- Updated dependencies [fe140c0]
- @kafkats/client@0.8.0

## 0.7.0

### Minor Changes
Expand Down
2 changes: 1 addition & 1 deletion packages/codec-zod/package.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"name": "@kafkats/codec-zod",
"private": false,
"version": "0.7.0",
"version": "0.8.0",
"description": "Zod codec helpers for @kafkats/client.",
"type": "module",
"main": "dist/index.js",
Expand Down
59 changes: 59 additions & 0 deletions packages/flow-state-lmdb/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,64 @@
# @kafkats/flow-state-lmdb

## 0.8.0

### Minor Changes

- fe140c0: ShareConsumer: Kafka 4.2 share groups GA support (ShareFetch/ShareAcknowledge v2)
- KIP-1206: new `acquireMode` config (`'batch_optimized'` default | `'record_limit'`) — `record_limit` strictly caps each fetch at `maxRecords`.
- KIP-1222: new `message.renew()` to extend the acquisition lock without finalizing delivery; safe to call multiple times per message.
- `ShareAcknowledgeResponse.acquisitionLockTimeoutMs` is now decoded on v2 responses.
- AckManager dedupes same-offset RENEW + finalizing-ack pairs and collapses duplicate same-offset RENEWs so the wire never carries overlapping batches.
- `@experimental` markers removed; share groups are GA in Kafka 4.2.
- Backward compatible: against Kafka 4.1 brokers the client negotiates v1 and produces byte-identical wire output to before this change. `record_limit` and `renew()` throw with a clear "requires v2 (Kafka 4.2+)" error against older brokers.

### Patch Changes

- ca7d22f: Correctness and reliability fixes across the producer, consumer, client protocol, and flow state stores.

### Producer
- `flush()` now waits for deferred drains so it no longer resolves while writes are still buffered.
- `transactionalSend` freezes its partition set up front to avoid double-calling the partitioner.
- Retry `TxnOffsetCommit` on retriable and rebalance-in-progress errors instead of failing the transaction.
- Fence the producer on `OUT_OF_ORDER_SEQUENCE_NUMBER` to surface the idempotence violation rather than silently corrupting the stream.
- Reject orphaned in-flight send promises during disconnect so callers are not left hanging.

### Consumer
- Always reschedule the heartbeat after non-fatal errors, preventing the consumer from dropping out of the group.
- Surface auto-commit errors to the consumer instead of swallowing them.
- Clear only the successfully committed partitions on a partial commit failure so the rest are retried.
- Fence stale fetch responses across partition replacement to avoid delivering records from a revoked assignment.

### Client / protocol
- Guard against varint shift overflow in the decoder.
- Close a `ConnectionPool.acquire` race that could hand out a connection mid-teardown.
- Clamp the SASL reauthentication delay to the `setTimeout` maximum.
- Don't await a response on `Produce` with `acks=0` (the broker sends none).
- Always use the v0 response header for `ApiVersions`, matching broker behavior across versions.

### Share consumer
- Release (rather than acknowledge) the last yielded message on shutdown so unprocessed records are redelivered.

### Flow
- Write to the local store before the changelog (local-first ordering) to shrink the inconsistency window on crash.
- Serialize the EOS commit triggered by a rebalance against the next message, and close the EOS rebalance offset-commit gap via an awaitable rebalance hook (shared with the consumer fix above).
- Use stream time (not wall-clock time) for window expiry.
- Implement hopping windows (`TimeWindows.advanceBy`).
- Throw on `SlidingWindows` instead of silently misbehaving (not yet implemented).
- Correct `TableGroupByNode` mapping ordering.
- Apply retention to `reduce` and session aggregators, emitting changelog tombstones for expired state.
- Surface checkpoint persistence errors instead of swallowing them.

### flow-state-lmdb
- Make `range()` upper bound inclusive, matching the in-memory store.
- Await `fsync` on checkpoint writes so a checkpoint is durable before it is reported complete.
- Correct `WindowStore` time-bound math.
- Use order-preserving signed-i64 encoding for time keys so range scans iterate in correct chronological order.

- Updated dependencies [ca7d22f]
- Updated dependencies [fe140c0]
- @kafkats/flow@0.8.0

## 0.7.0

### Minor Changes
Expand Down
4 changes: 2 additions & 2 deletions packages/flow-state-lmdb/package.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"name": "@kafkats/flow-state-lmdb",
"private": false,
"version": "0.7.0",
"version": "0.8.0",
"description": "LMDB state store provider for @kafkats/flow",
"type": "module",
"main": "dist/index.js",
Expand Down Expand Up @@ -48,7 +48,7 @@
"lmdb": "^3.2.6"
},
"peerDependencies": {
"@kafkats/flow": "workspace:>=0.7.0"
"@kafkats/flow": "workspace:>=0.8.0"
},
"devDependencies": {
"@kafkats/flow": "workspace:^",
Expand Down
Loading