fix(kafka source): prevent data loss on rejected messages with acknowledgements #24557
+172
−5
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Fixes data loss in the Kafka source when acknowledgements are enabled and message delivery fails.
Problem
When a message delivery fails (e.g., due to authentication errors) with acknowledgements enabled, the Kafka source would skip the failed message instead of retrying it, causing data loss.
The issue occurred because:
Solution
This PR ensures rejected messages are retried by:
last_committed_offseton the first message to enable retry even for the first messageseek_to_retry_offset()method with proper error handlingsocket_timeout_msfor seek operations,fetch_wait_max_msfor retry delay)Changes
last_committed_offsettracking to remember last successful commitneed_seek_backflag to trigger retry mechanismDeliveredvsErrored/Rejectedseek_to_retry_offset()method for clean separation of concernsseeks_back_on_rejected_messageto validate the fixTesting
seeks_back_on_rejected_messagevalidates the retry mechanismThe integration test verifies:
Checklist