Skip to content

datastore: apply schema changes immediately to committed state. #2685

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

Centril
Copy link
Contributor

@Centril Centril commented Apr 30, 2025

Description of Changes

TODO

API and ABI breaking changes

None

Expected complexity level and risk

4, this is isolated to the datastore, but the datastore is critical, subtle, and contains various unsafe.

Testing

TODO

@Centril Centril requested a review from Copilot April 30, 2025 15:54
Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR introduces immediate application of schema changes to the committed datastore state and updates various index and schema manipulation APIs. Key changes include the addition of new merging functions (can_merge) for several index types, revised schema mutation and index deletion APIs in Table, and significant restructuring of transaction state and rollback mechanisms in the core datastore layer.

Reviewed Changes

Copilot reviewed 14 out of 14 changed files in this pull request and generated no comments.

Show a summary per file
File Description
crates/table/src/table_index/uniquemap.rs Added a can_merge method to verify if two UniqueMaps can be merged
crates/table/src/table_index/unique_direct_index.rs Added a can_merge method for unique direct indices using a zip iteration over inner collections
crates/table/src/table_index/unique_direct_fixed_cap_index.rs Added a can_merge function to similar effect for fixed capacity indices
crates/table/src/table_index/mod.rs Introduced a could_merge method for TableIndex that dispatches to the appropriate can_merge functions
crates/table/src/table.rs Modified schema mutation functions and index add/delete operations with updated pointer map handling
crates/schema/src/schema.rs Revised adjacent schema modifications using a new take_adjacent_schemas API and helper find_remove
crates/core/src/db/datastore/locking_tx_datastore/* Updated transaction state and rollback handling to immediately reflect schema changes, plus various related API adjustments
Comments suppressed due to low confidence (3)

crates/table/src/table_index/uniquemap.rs:87

  • Add unit tests for the new can_merge method to verify its behavior when merging maps that have conflicting keys as well as when no conflicts are present.
pub(crate) fn can_merge(&self, other: &UniqueMap<K, V>) -> Result<(), &V> {

crates/table/src/table_index/unique_direct_index.rs:210

  • Ensure that unit tests are added for can_merge to cover scenarios where inner slot conflicts occur and where merge should succeed without errors.
pub(crate) fn can_merge(&self, other: &UniqueDirectIndex) -> Result<(), RowPointer> {

crates/table/src/table_index/unique_direct_fixed_cap_index.rs:126

  • Introduce tests for the new can_merge function to confirm that the function correctly identifies merge conflicts when both slots are occupied.
pub(crate) fn can_merge(&self, other: &UniqueDirectFixedCapIndex) -> Result<(), RowPointer> {

@Centril Centril force-pushed the centril/datastore-revamp-schema-changes branch from 4cacd67 to a3ecb73 Compare April 30, 2025 16:47
@Centril Centril force-pushed the centril/datastore-revamp-schema-changes branch from a3ecb73 to 3661d73 Compare April 30, 2025 18:19
@CLAassistant
Copy link

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants