Skip to content

Conversation

@michael-redpanda
Copy link
Contributor

@michael-redpanda michael-redpanda commented Feb 3, 2026

Clarifies that the delay prevents data loss when the source partition performs prefix truncation in the middle of a batch. The replicator needs to replicate the entire batch before the shadow partition can safely truncate.

  • Run test 1000 times

Backports Required

  • none - not a bug fix
  • none - this is a backport
  • none - issue does not exist in previous branches
  • none - papercut/not impactful enough to backport
  • v25.3.x
  • v25.2.x
  • v25.1.x

Release Notes

  • none

Clarifies that the delay prevents data loss when the source partition
performs prefix truncation in the middle of a batch. The replicator
needs to replicate the entire batch before the shadow partition can
safely truncate.
@michael-redpanda michael-redpanda self-assigned this Feb 3, 2026
Copilot AI review requested due to automatic review settings February 3, 2026 21:10
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR improves code clarity by adding a comprehensive comment explaining the delay logic for prefix truncation in cluster link replication. The enhancement addresses the scenario where a source partition performs prefix truncation that lands in the middle of a batch, which could lead to data loss if the shadow partition truncates before the replicator finishes replicating the entire batch.

Changes:

  • Added detailed comment explaining why prefix truncation must be delayed until the shadow partition has fully replicated batches up to or past the source start offset
  • Introduced local variables (shadow_partition_hwm and source_lso) to make the logic more readable and support the explanation in the comment

@michael-redpanda
Copy link
Contributor Author

/ci-repeat 10
release
debug
skip-unit
dt-repeat=100
skip-redpanda-build
tests/rptest/tests/cluster_linking_e2e_test.py::ShadowLinkingReplicationTests.test_auto_prefix_trimming

@vbotbuildovich
Copy link
Collaborator

CI test results

test results on build#80087
test_class test_method test_arguments test_kind job_url test_status passed reason test_history
ShadowLinkingReplicationTests test_auto_prefix_trimming {"source_cluster_spec": {"cluster_type": "redpanda"}, "with_failures": true} integration https://buildkite.com/redpanda/redpanda/builds/80087#019c28fc-8713-40b2-8140-3bb0a67ee60a FLAKY 109/110 Test PASSES after retries.No significant increase in flaky rate(baseline=0.0017, p0=1.0000, reject_threshold=0.0100. adj_baseline=0.1000, p1=0.3487, trust_threshold=0.5000) https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=ShadowLinkingReplicationTests&test_method=test_auto_prefix_trimming
ShadowLinkingReplicationTests test_auto_prefix_trimming {"source_cluster_spec": {"cluster_type": "redpanda"}, "with_failures": true} integration https://buildkite.com/redpanda/redpanda/builds/80087#019c28fc-8713-41bd-a0c8-24f0dc52ea96 FLAKY 109/110 Test PASSES after retries.No significant increase in flaky rate(baseline=0.0017, p0=1.0000, reject_threshold=0.0100. adj_baseline=0.1000, p1=0.3487, trust_threshold=0.5000) https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=ShadowLinkingReplicationTests&test_method=test_auto_prefix_trimming

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants