feat(palace): auto-migrate legacy metadata on every read/write (mp-migration 24)#24
Merged
Merged
Conversation
…ments/superseded_by/active to first-class Drawer fields
The new fields mirror values previously stored only in the
`metadata` HashMap under the keys 'tags', 'trust', 'access_count',
'last_accessed', 'reinforcements', 'superseded_by', 'active'. They
are now typed first-class fields for:
- type-safe access from the upcoming MemoryProvider trait methods
(boost/decay/reinforce/supersede/tag/link in mp-migration 1+2/8)
- clean serde round-trip without manual value extraction
- query-builder support (drawer.tags([...]).trust('high'))
`#[serde(default, skip_serializing_if = ...)]` on every new field
keeps backwards compatibility — drawers serialised before this change
still load cleanly. The reverse direction (writing the typed field)
is handled by `Drawer::migrate_metadata` which is idempotent and
safe to call repeatedly; it lifts legacy metadata keys into typed
fields when those fields are still at their default value, and
cleans the metadata HashMap so the source-of-truth is unambiguous.
`Reinforcement` is promoted to a pub struct in palace.rs (mirrors
jcode's `memory_types::Reinforcement`). `DrawerId` is reused for
`superseded_by` to keep the ID type consistent across the API.
Existing call sites in:
- crates/core/src/palace/store/usearch_sqlite.rs (Drawer literal x2)
- crates/core/src/layers.rs:652 (Drawer literal x1)
updated to initialise the new fields with their defaults.
This is PR 7/8 in the jcode → mempalace Mode C library migration
series. Required for PR 1 (mutation methods) and PR 2 (tag/link).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
PR #7 added the new typed Drawer fields (tags, trust, access_count, last_accessed, reinforcements, superseded_by, active) plus a `migrate_metadata` helper. The helper was left as opt-in (consumers had to call it themselves), which meant legacy drawers written by pre-PR #7 Palace versions would still be read in the v0 shape (data in `metadata`, typed fields empty). This PR wires the migration into every read/write path so the self-healing is automatic and the typed fields are the source of truth on disk from this point forward. Wiring: - `Palace::add_drawer` (palace.rs:625) — calls `drawer.migrate_metadata()` once at the top, before any `store.upsert`. Every `add_drawer` write is now a self-healing write: legacy drawers that get re-written land in the v1 (typed-field) shape automatically. - `Palace::get_drawers` (palace.rs:710) — iterates the returned drawers and calls `migrate_metadata` on each before returning. Every reader (Layer 1 wake-up, status, PR #1 mutation methods via `default_mutate_drawer`, PR #2 tag/link, PR #4 recent) flows through this single chokepoint, so the migration propagates through the whole read surface. - `palace/store/usearch_sqlite.rs::get_drawer_by_id` and `::all_drawers` — call `migrate_metadata` right after constructing the Drawer from SQLite rows. Defensive: covers the case where a host calls `store.get_drawers()` directly (bypassing `Palace::get_drawers`). - `layers.rs::Layer1::generate` and `layers.rs::create_test_palace_db` (palace_db.rs read path) — same defensive coverage for the wake-up layer and tests that go through `PalaceDb::get_all` rather than the trait. The `migrate_metadata` function is left as `pub fn` (not demoted to `pub(crate)`) to avoid a SemVer-breaking change on PR #7. Consumers can still call it explicitly if they want to; they just don't have to. Includes a cherry-pick of PR #7 (the `migrate_metadata` helper and Drawer field extension) so this branch has the full self-healing pipeline. The cherry-pick is a separate commit so the two PRs are reviewable independently. This is PR 24/8+ (post-series) — closes the design gap in PR #7 that was called out in the PR #7 review: migrate_metadata should be mempalace's responsibility, not the consumer's. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
quangdang46
force-pushed
the
feat/mp-pr24-auto-migrate-metadata
branch
from
01f60a5 to
eb9a6bd
Compare
4 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Wires
Drawer::migrate_metadata(added in PR #7) into every read and write path in mempalace so legacy drawers (pre-PR #7, with data only inmetadata) are auto-migrated to the v1 typed-field shape. After this PR, the typed fields are the source of truth on disk from this point forward, and consumers (jcode's adapter, third-party agents) never have to think about the migration.Motivation
PR #7 added the new typed
Drawerfields and amigrate_metadatahelper, but left the helper as opt-in (consumers had to call it themselves). This was a design gap:metadata["tags"]shape.This PR closes the gap by making the migration automatic at the data boundary.
Change
Palace::add_drawer(palace.rs:625) — callsdrawer.migrate_metadata()once at the top, before anystore.upsert. Every write is now a self-healing write: legacy drawers that get re-written land in the v1 shape.Palace::get_drawers(palace.rs:710) — iterates the returned drawers and callsmigrate_metadataon each before returning. This is the single chokepoint that most reads flow through:boost,decay,reinforce,supersede,set_metadata) viadefault_mutate_drawertag,untag,link,list_tagsviadefault_mutate_drawerrecent(retention-ranked recall)palace/store/usearch_sqlite.rs—get_drawer_by_idandall_drawerscallmigrate_metadataright after constructing theDrawerfrom SQLite rows. Defensive coverage for hosts that callstore.get_drawers()directly (bypassing the trait).layers.rs::Layer1::generate— same defensive coverage for the wake-up layer, which reads viaPalaceDb::get_all(returnsQueryResult, notDrawer) and constructs theDrawerstruct inline.migrate_metadatavisibility — left aspub fn(not demoted topub(crate)) to avoid a SemVer-breaking change to PR #7's surface. Consumers can still call it explicitly if they want; they just don't have to.Includes a cherry-pick of PR #7 (the
migrate_metadatahelper and Drawer field extension) so this branch has the full self-healing pipeline. The cherry-pick is a separate commit so the two PRs remain reviewable independently.Test plan
cargo test -p mempalace-core --lib→ 1133 passed, 0 failed (953s, includes the 6 tests from PR fix: port upstream symlink logging and MCP unknown-param error (#1462, #1512) #7'smigrate_metadata_*family)cargo check -p mempalace-corecleancargo fmt --check -p mempalace-corecleanDiff size
3 files changed, 48 insertions(+), 16 deletions(-)
palace.rs: addmigrate_metadata()call at the top ofadd_drawer(1 line + comment), iterate + migrate inget_drawers(3 lines + comment)palace/store/usearch_sqlite.rs: convert struct-literal}to;(let-binding), addmigrate_metadata()call (1 line + comment) × 2 functionslayers.rs: same let-binding + migrate pattern inLayer1::generate(3 lines + comment)Series status
🤖 Generated with Claude Code