Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/en/docs/operate/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ Start with the operational task you need:
| --- | --- |
| Keep a personal Server running, deploy a container, or enable authentication | [Deploy the Server](deploy-server.md) |
| Upgrade or switch background supervision mode | [Migrate Artifact processing state](artifact-processing-migration.md) |
| Enable bounded Memory-entry queries after upgrading an existing database | [Migrate the Memory query index](memory-query-migration.md) |
| Inspect logs, metrics, and traces | [Observability](observability.md) |
| Send traces to an analysis service | [Phoenix](trace-with-phoenix.md), [Langfuse](trace-with-langfuse.md) |
| Diagnose failures and restore service or data | [Troubleshoot](troubleshoot.md) |
Expand Down
59 changes: 59 additions & 0 deletions docs/en/docs/operate/memory-query-migration.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
---
title: Migrate the Memory query index
---

The bounded Memory-entry query uses a rebuildable directory projection. A new
database is marked ready automatically. After upgrading a database that already
contains Memory revisions, the Server keeps legacy list, exact-detail, search,
and write operations available, but the new query returns
`memory_query_index_unavailable` until this migration is verified.

## Plan and apply

Use the same deployment environment file as the Server:

```shell
powercontext server memory-query-migrate --action plan --env-file .env
```

The plan is read-only and reports missing projection tables, authoritative
Memory revision count, existing directory rows, and the durable phase.

Back up the database. Stop every old Server and Memory writer, disable their
automatic restart, and keep them stopped for the maintenance window. The
command cannot establish those external conditions. Then run:

```shell
powercontext server memory-query-migrate --action apply --env-file .env \
--migration-id rfc1656 --batch-size 100 --maintenance-confirmed
powercontext server memory-query-migrate --action verify --env-file .env
```

`apply` creates any missing feature tables one at a time and commits at most the
selected number of Memory revisions per backfill transaction. It verifies the
result before reporting success. If it stops, repeat it with the same migration
ID. Do not clear the marker or directory rows; a different ID is rejected while
an earlier migration is incomplete.

The batch size counts revisions, not entries inside a revision. Rebuilding one
large initial manifest therefore writes one derived directory row per entry in
that transaction. Those rows use set reads and executemany writes in chunks of
500; choose the revision batch size and maintenance window with the largest
stored manifest in mind.

Only restart writers and Server replicas after the command reports
`ready: true`. The separately runnable `verify` action rechecks every
authoritative revision and tag-generation row and is safe to repeat.

## What the migration changes

The migration reads immutable Memory manifests and rebuilds revision-valid
compact directory rows plus one tag generation per Memory Artifact. It never
rewrites Memory Artifacts, entry versions, tags, or entry bodies. The directory
remains derived state and is not used as an authority until full verification
succeeds.

Memory writes from the new version maintain their own directory deltas, but a
rolling deployment with old and new writers during this offline migration is
unsupported. SQLite migration requires a persistent database; in-memory SQLite
has no upgrade state to preserve.
1 change: 1 addition & 0 deletions docs/en/docs/operate/meta.json
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
"deploy-server",
"connect-remote-server",
"artifact-processing-migration",
"memory-query-migration",
"observability",
"trace-with-phoenix",
"trace-with-langfuse",
Expand Down
4 changes: 2 additions & 2 deletions docs/en/docs/operate/troubleshoot.md
Original file line number Diff line number Diff line change
Expand Up @@ -269,7 +269,7 @@ so the previous database remains available for recovery:

```bash
obloader <connection-options> -D <new-database> --csv \
--table 'pc_scopes,pc_source_journal_heads,pc_sources,pc_artifacts,pc_source_cursors,pc_artifact_processing_leases,pc_artifact_processing_binding_states,pc_artifact_processing_pending,pc_artifact_processing_auto_wave_targets,pc_artifact_processing_sequences,pc_artifact_processing_intents,pc_topic_memory_processing_targets,pc_artifact_processing_schema,pc_artifact_processing_migration_receipts,pc_topic_memory_work_budgets,pc_topic_memory_retrieval_shape,pc_connector_checkpoints,pc_source_definition_manifests,pc_external_skill_registrations,pc_skill_packages,pc_agent_skill_targets,pc_skill_publications,pc_model_usage_daily,pc_recall_token_daily,pc_receipt_migration_review' \
--table 'pc_scopes,pc_source_journal_heads,pc_sources,pc_artifacts,pc_memory_query_index_schema,pc_source_cursors,pc_artifact_processing_leases,pc_artifact_processing_binding_states,pc_artifact_processing_pending,pc_artifact_processing_auto_wave_targets,pc_artifact_processing_sequences,pc_artifact_processing_intents,pc_topic_memory_processing_targets,pc_artifact_processing_schema,pc_artifact_processing_migration_receipts,pc_topic_memory_work_budgets,pc_topic_memory_retrieval_shape,pc_connector_checkpoints,pc_source_definition_manifests,pc_external_skill_registrations,pc_skill_packages,pc_agent_skill_targets,pc_skill_publications,pc_model_usage_daily,pc_recall_token_daily,pc_receipt_migration_review' \
-f <export-directory>
```

Expand All @@ -285,7 +285,7 @@ so the previous database remains available for recovery:

```bash
obloader <connection-options> -D <new-database> --csv \
--table 'pc_artifact_candidate_heads,pc_topic_memory_active_topics,pc_topic_memory_active_chunks,pc_memory_entry_heads,pc_artifact_tags,pc_recurrence_match,pc_recurrence_observation' \
--table 'pc_artifact_candidate_heads,pc_topic_memory_active_topics,pc_topic_memory_active_chunks,pc_memory_entry_heads,pc_memory_entry_directory,pc_memory_tag_generations,pc_artifact_tags,pc_recurrence_match,pc_recurrence_observation' \
-f <export-directory>
```

Expand Down
Loading
Loading