Skip to content

fix(protocol): backport ElectLeaders V1 non-flexible header fix#16

Merged
3AceShowHand merged 2 commits intopingcap:v1.41.2-pingcapfrom
3AceShowHand:fix-elect-leaders
Apr 20, 2026
Merged

fix(protocol): backport ElectLeaders V1 non-flexible header fix#16
3AceShowHand merged 2 commits intopingcap:v1.41.2-pingcapfrom
3AceShowHand:fix-elect-leaders

Conversation

@3AceShowHand
Copy link
Copy Markdown
Collaborator

@3AceShowHand 3AceShowHand commented Apr 20, 2026

Summary

Backport the upstream ElectLeaders protocol fixes to v1.41.2-pingcap and align our flexible protocol codec with upstream so that low-version ElectLeaders requests and responses use the correct header format.

Problem

After #3209 was backported, our branch sat in a half-migrated state for flexible protocol handling:

  • request/response header selection and body encoding were not fully aligned with upstream flexible codec behavior
  • ElectLeaders V1 could be treated as flexible when it must use the non-flexible header format
  • this can break compatibility for lower-version clients talking to newer brokers, with failures like:

kafka: insufficient data to decode packet, more bytes expected

What This PR Does

  • backport the upstream flexible encoding/decoding infrastructure needed by these protocol paths
  • backport the supporting decoder fix used by that infrastructure
  • fix ElectLeaders request/response header version selection so V1 uses non-flexible headers and V2 uses flexible headers
  • add request/response regression tests for ElectLeaders V1 and V2

Why The Diff Is Larger Than The Upstream Fix PR

The direct upstream ElectLeaders fix is small, but our branch did not yet fully match the upstream flexible codec architecture. To make the backport correct and reduce future cherry-pick conflicts, this PR also brings over the supporting codec changes that upstream already relies on.

Upstream References

Primary upstream PR:

Related upstream fix history:

Supporting upstream commits backported here:

  • 7b11d9cc refactor: flexible decoding
  • 78601bd8 refactor: flexible encoding
  • 9e0c1c29 fix: decoder issues (#3327)

Upstream issue:

  • I did not find a separate upstream issue for this ElectLeaders compatibility bug; IBM/sarama#3478 itself documents the problem and repro.

Testing

Verified locally with:

  • env GOCACHE=/tmp/go-build-cache go test ./... -run '^$'
  • env GOCACHE=/tmp/go-build-cache go test . -run '^TestElectLeaders(Request|Response|RequestHeaderVersion|ResponseHeaderVersion)$' -count=1
  • targeted protocol/unit tests covering the protocol files touched by the flexible codec backport

@3AceShowHand 3AceShowHand changed the title fix api version fix(protocol): backport ElectLeaders V1 non-flexible header fix Apr 20, 2026
Backport IBM#3478 and the supporting flexible codec changes\nneeded on v1.41.2-pingcap.\n\nThis fixes ElectLeaders V1 request/response header selection for\nlow-version clients talking to newer brokers.
@3AceShowHand 3AceShowHand merged commit 05c84a4 into pingcap:v1.41.2-pingcap Apr 20, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant