Skip to content

feat(artifacts): share merge and restoration across families - #1880

Merged
hidb4ai merged 59 commits into
oceanbase:masterfrom
frf12:codex/artifact-merge
Oct 11, 2026
Merged

hidb4ai merged 59 commits into
oceanbase:masterfrom
frf12:codex/artifact-merge

Conversation

@frf12

@frf12 frf12 commented Oct 8, 2026 •

Copy link
Copy Markdown
Member

Which issue or RFC does this PR close?

Implements RFC #1859 and builds on RFC #1809 and Atomic Memory implementation #1857.

RFCs #1809 and #1859 and implementation #1857 are merged. This branch includes master at 7dfa9031793e5fb4b737cc3bed20b03c17f1ce3e, the squash commit for #1857. Keep this implementation in the same release batch as #1857 so the private Atomic Memory state-table format is never released.

Rationale for this change

Let Artifact Families share merge and restoration persistence through the existing Artifact heads and lineage tables. Each Family retains the business decision of whether and what to merge, without creating its own merge-specific tables.

What changes are included in this PR?

  • Add a common merge/restoration service with Family adapters for Atomic Memory and Experience, plus coverage for registration of another Family.
  • Store merge destinations in pc_artifact_heads.merged_into_id and identify merge-input relationships with pc_artifact_lineage_artifacts.is_merge_input. Atomic Memory uses shared heads instead of its private state table.
  • Restore complete merge groups, including cascading undo. Each Artifact actually restored or retired receives a new revision under its existing identity; old revisions and original merge-input relationships remain immutable. Requests with no affected state and no explicitly selected revision are no-ops.
  • Persist each restoration group's exact before/content-source/after references and final states in an existing Content Source, atomically with the revisions, heads and projections. Bind it only to the primary new revision and authorize every affected exact Artifact reference when reading the outcome. No separate operation ledger or access-audit schema extension is added.
  • Preserve shared access authorization, legacy Memory migration and current-record eligibility. Integrate recall-gate and statistics changes, including detection of an existing candidate gaining a second retrieval channel during recall expansion.

Are there any user-facing changes?

runtime.artifact_merge.for_scope(scope_id, family) exposes get, merge, preview_restoration, restore, and restoration_outcome. Experience gains explicit Python merge/restoration support. Atomic Memory keeps its Python/HTTP adapter; this change does not add a generic HTTP/MCP endpoint or an automatic Experience merge policy.

Successful restore/undo responses identify newly appended revisions. restoration_outcome supports an exact historical revision and returns None if no outcome exists.

Startup upgrades the existing shared tables. Legacy Memory migration remains supported, including read-only planning/verification against the old schema and preserving evolved heads on replay. The unreleased pc_atomic_memory_states format is rejected rather than migrated; the release ordering above is required.

How was this change tested?

Current head: cee50bc1a19d3be49f103f660c81c6740990c4cb, with merged #1857/master commit 7dfa9031793e5fb4b737cc3bed20b03c17f1ce3e as an ancestor. The committed tree matches the independently constructed expected result: the previously reviewed #1880 tree plus #1857's final five-file delta. Shared merge/restoration code and the existing evolved-head migration regression are preserved.

  • make check: release/action pins, lockfile, all 11 hooks, package/integration type checks and 49 integration-manifest tests passed.
  • Full access-adapter, access-control, migration-evidence, migration-CLI and startup-schema test files: 98 passed, 0 failed, 52 skipped, 0 deselected. The skips require opt-in SeekDB/OceanBase migration backends unavailable locally.
  • Independent shared merge/restoration, storage, registration, cascading undo and permission-snapshot regression: 44 passed, 0 failed, 9 skipped, 0 deselected. The nine skips require OceanBase.
  • Source review confirms custom Provider/Repository rules remain active during transactional merge, restoration and outcome reads. The migration check distinguishes structured legacy references from ordinary text mentioning legacy field names.

The preceding #1880 head fefc285a completed 30 successful checks and one conditional skip. Fresh checks for the current merge commit are running; the preceding head's results are not presented as current-head CI success.

Validation boundaries: local runs use SQLite and controlled inference, not live backend/model/host acceptance. Earlier real OceanBase/SeekDB runs do not establish coverage of this head. Earlier live generation/host acceptance remains 14 passed / 12 failed (model-output assertions, Journey harness, SQLite vec0 cleanup and ZCode controlled-harness/SSE failures); these have not been established as fixed. Complete live-model/host acceptance is not claimed.

AI usage statement

Implemented and validated with OpenAI Codex using GPT-6.1 Sol subagents at Ultra reasoning effort. The primary agent reviewed the changes and independently checked source scope and validation evidence under maintainer direction.

@frf12
frf12 marked this pull request as ready for review October 8, 2026 03:24
Comment thread src/powercontext/builtin/artifacts/merge.py
frf12 and others added 2 commits October 10, 2026 12:01
Runtime and persistence tags only address Artifacts; the deterministic
legacy identity mapping stays in the server get_memory_entry adapter.
The server rejects legacy collection tags and tag queries that select
the memory family or memory_entry targets, and default tag queries
exclude legacy Memory.

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@hidb4ai

hidb4ai commented Oct 10, 2026

Copy link
Copy Markdown
Contributor

please update & resolve conflicts

frf12 and others added 6 commits October 10, 2026 17:36
- Archive and remove whole-collection citations in Handoffs and Work, blocking
  when a claim loses its required evidence.
- Remove all collection-owned lineage before deleting collections.
- Let an applied decision file be reused on rerun.
- Block unfinished Dream runs whose pinned Artifacts or Sources change.
- Accept import, references and projection before removal; startup runs only
  a light residual check.
- Recreate development current-projection layouts during apply.

Co-authored-by: Cursor <cursoragent@cursor.com>
…ntract

The runtime, public models, OpenAPI contract, SDK, dashboard and integrations
no longer know the legacy Memory collection: MemoryService, its persistence
and indexes, MemoryCitation, memory_citations fields and Handoff memory
citations are removed. ArtifactRef rejects family=memory, so new requests
cannot reintroduce collection references; generic reads and tag queries of
family=memory are unsupported.

atomic-memory-migrate gains the remaining data steps and stays repeatable:
- Handoff receipt evidence citing a whole collection moves to the receipt's
  historical_data;
- unfinished Dream runs drop the empty entry citation fields, and request
  digests are recomputed so identical retries still replay;
- Task Outcome item digests are recomputed from frozen legacy decoders;
- the emptied memory_citations columns are dropped last, and startup refuses
  while they exist.

The legacy write gate stays rejected at configuration time. The legacy
collection capacity benchmark is removed. Desktop reads only Atomic Memory
references and no longer bundles citation-only Server profiles.

Co-Authored-By: Claude Code <noreply@anthropic.com>
Co-Authored-By: Claude Code <noreply@anthropic.com>
@frf12
frf12 requested a review from knqiufan as a code owner October 10, 2026 13:25
frf12 and others added 3 commits October 10, 2026 21:35
Recall-gate relevance (oceanbase#1918) reads Atomic Memory hits' vector distance as
cosine relevance. Its e2e expectations follow Atomic Memory search, which
embeds kind and body together, limits each channel before fusion and reports
recoverability as an existence probe rather than admission counts.

Co-Authored-By: Claude Code <noreply@anthropic.com>
… citations

Co-Authored-By: Claude Code <noreply@anthropic.com>
@hidb4ai

hidb4ai commented Oct 10, 2026

Copy link
Copy Markdown
Contributor

ci falied

@hidb4ai hidb4ai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed d233cc8 with real Qwen/OceanBase execution and isolated SQLite/seekDB tests. The migration defect below remains blocking: an actual SQLite configuration reproduces the five migration failures seen in Python 3.13 CI.

Comment thread src/powercontext/builtin/persistence/migrations/atomic_memory_references.py Outdated

@hidb4ai hidb4ai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reviewed fefc285. The reported binary-JSON migration defect is fixed: the original counterexamples now pass with native SQLite 3.50.2 compiled with SQLITE_LIKE_DOESNT_MATCH_BLOBS. I also checked the new migration/readiness/receipt/FTS changes and reran real OceanBase and native seekDB migration, grant replay/conflict, and vector rebuild scenarios, plus the TCP MCP workflow with real embeddings. No remaining blocking findings in the reviewed scope.

Some CI jobs were still running at the final check. These checks cover the reviewed behavior; full installed-host and release acceptance remain outside this review.

@hidb4ai hidb4ai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@hidb4ai
hidb4ai merged commit 3b58d49 into oceanbase:master Oct 11, 2026
31 checks passed

@hidb4ai hidb4ai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reviewed cee50bc after the upstream/master sync. The original migration counterexamples still pass with native SQLite 3.50.2 compiled with SQLITE_LIKE_DOESNT_MATCH_BLOBS. I verified the parsed-field residual checks on real OceanBase and native seekDB: ordinary Handoff/receipt text mentioning memory_citation stays unchanged, while actual legacy references are converted and the migrated Handoff remains readable. Custom-provider denial, authorization boundary, and shared merge/restoration regressions also pass. No remaining blocking findings in the reviewed scope.

Some CI jobs were still running at the final check; this approval does not claim completed release or installed-host acceptance.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants