For RSE, we want the invariant that command evaluation at the leaseholder produces batches that can only touch state machine engine keys. Or, easier to enumerate (there are only 3), it should not touch the raft engine keys.
|
// 2. Unreplicated range-ID local keys: These contain metadata that |
|
// pertain to just one replica of a range. They are unreplicated and |
|
// unaddressable. The typical example is the Raft log. They all share |
|
// `LocalRangeIDPrefix` and `localRangeIDUnreplicatedInfix`. |
|
// |
|
// WARNING: when adding a new key in this section, decide whether it should be |
|
// classified as "raft" or "state machine" key, correspondingly to which |
|
// engine it resides in: |
|
// |
|
// - keys <= RangeTombstoneKey in this prefix are "state machine" engine keys |
|
// - keys > RangeTombstoneKey in this prefix are "raft" engine keys |
|
// - historical exception: RaftReplicaIDKey belongs to the state machine |
|
// |
|
// Failure to classify may result in replica state corruption in storage. |
|
localRangeIDUnreplicatedInfix, // "u" |
|
RangeTombstoneKey, // "rftb" |
|
RaftHardStateKey, // "rfth" |
|
RaftLogKey, // "rftl" |
|
RaftReplicaIDKey, // "rftr" |
|
RaftTruncatedStateKey, // "rftt" |
|
RangeLastReplicaGCTimestampKey, // "rlrt" |
We know of 2 exceptions when raft engine keys are written into the eval batch, and work around them. For confidence, it would be good to confirm that there no other exceptions, or find all.
The mechanism to do that would be similar to the spanset.Batch here, which asserts on keys in relation to the latches:
|
if util.RaceEnabled { |
|
// During writes we may encounter a versioned value newer than the request |
|
// timestamp, and may have to retry at a higher timestamp. This is still |
|
// safe as we're only ever writing at timestamps higher than the timestamp |
|
// any write latch would be declared at. But because of this, we don't |
|
// assert on access timestamps using spanset.NewBatchAt. |
|
batch = spanset.NewBatch(batch, g.LatchSpans()) |
|
} |
Epic: CRDB-55220
Jira issue: CRDB-56053
