Repository navigation
Conversation
… coordinator after partition leadership change (apache#22466) During routine `__consumer_offsets` partition leadership changes, the group coordinator spams ERROR-level logs for every in-flight write at the moment of transition: ``` [GroupCoordinator id=N] Writing records to __consumer_offsets-N failed due to: For requests intended only for the leader, this error indicates that the broker is not the current leader ... [GroupCoordinator id=N] Execution of FlushBatch failed due to For requests intended only for the leader, this error indicates that the broker is not the current leader ... ``` These appear on the group coordinator that lost leadership and last for the duration of the in-flight batch queue. The behavior is correct: `NotLeaderOrFollowerException` propagates through `failCurrentBatch` to the deferred events and is mapped to `NOT_COORDINATOR` for clients via `CoordinatorOperationExceptionHelper`, so clients retry against the new coordinator. This is purely a logging-noise issue. This patch classifies the flush failures in `flushCurrentBatch`: - `NotLeaderOrFollowerException` (leadership change) and `InvalidProducerEpochException` (fenced producer) are logged at DEBUG; - `RecordTooLargeException` and `RecordBatchTooLargeException` remain at ERROR with added context (record count and maximum batch size); - `NotCoordinatorException`, thrown when the state machine is out of sync, is not re-logged since that condition is already logged; - any other failure is logged at ERROR as an unexpected error. The redundant re-log from the `FlushBatch` event is also suppressed so that a failure is logged once. Reviewers: Dongnuo Lyu <dlyu@confluent.io>, Sean Quah <squah@confluent.io>
dajac
changed the title
squah-confluent
approved these changes
Jun 8, 2026
Member
|
I assume 4.0 is archived since there are 4.1, 4.2, and 4.3 |
Member
Author
|
As CI is broken in 4.0, I will cherry-pick it manually from 4.1 just in case we would ever release a new version of 4.0. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Backport of #22466 to 4.0.
This branch predates the adaptive-flush refactor, so the change is adapted:
flushCurrentBatchlog classification and thecompleteTransactiontransaction-marker logging are applied as in the original;
withActiveContexthelper is added and the single (linger)FlushBatchevent uses it (there is noenqueueAdaptiveFlushhere);maxBatchSizefield onCoordinatorBatch(the original patch added that field on trunk, but italready exists on this branch).
Compiled and
CoordinatorRuntimeTestpasses on the branch.