catalog: Key migration shard by binary version #31210
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The builtin migration shard is a persist shard used during version
upgrades. It uses the environment's deploy generation as part of the
key for the values. The assumption was that two environments with the
same deploy generation would always have the same binary version. This
assumption would allow all migration steps of two environments with the
same deploy generation to be idempotent.
This assumption was not correct. Two environments with different binary
version can use the same deploy generation as long as one environment
never fully completed a deployment. This is especially bad because the
migration shard is written to and read from in read-only mode, before
a deployment is complete.
This commit updates the key of the builtin migration shard to
explicitly use the binary version of environmentd so that the
migration steps are idempotent.
Fixes #MaterializeInc/database-issues/issues/8917
Motivation
This PR fixes a previously unreported bug.
Checklist
$T ⇔ Proto$T
mapping (possibly in a backwards-incompatible way), then it is tagged with aT-proto
label.