Repository navigation
bug(memory-migration): residual Source check crashes on payloads stored before the source envelope #1934
Description
Describe the bug
atomic-memory-migrate --action apply aborts with KeyError: 'value' when the database contains content Sources that were stored before the powercontext-source-v1 payload envelope existed.
residual_issues() narrows candidate rows with a free-text match, CAST(payload AS CHAR) LIKE '%memory%', and then unconditionally reads json.loads(payload)["value"]. Sources written by an older release store the source value directly ({"content": ..., "description": ..., "materialization": ..., "metadata": ..., "name": ...}) and have no value key, so any captured user prompt or Source whose text merely contains the substring memory crashes the migration.
The runtime read path already tolerates both layouts through _decode_legacy_payload(), so the offline checks are the only places that assumed the envelope unconditionally.
Correction after review: the conversion path is affected too. _work_sources() and _dream_inputs_change() read the same field, so a legacy structured Work Source aborts the conversion step or makes an unfinished Dream run look unaffected. _work_sources() narrows with LIKE '%"memory%', which does match an escaped \"kind\": \"memory\" in a stored payload, so the earlier claim that only the residual check is affected was wrong.
Steps to reproduce
- Run an older PowerContext release with prompt capture enabled and let it store content Sources (for example
opencode-user-prompt:<hash>) whose text contains the wordmemory. - Upgrade to a build that includes
feat(memory): implement independent atomic memory(feat(memory): implement independent atomic memory #1857) and start the Server against that database. - Startup fails with
AtomicMemoryMigrationErrorreporting that legacy Memory collections remain in public Artifact tables. - Back up the database, stop all writers and run
powercontext server atomic-memory-migrate --action apply --env-file .env --maintenance-confirmed. - Apply imports every entry and then aborts before removing any collection:
File "src/powercontext/builtin/persistence/migrations/atomic_memory_v1.py", line 1307, in apply_atomic_memory_migration
report = await _verify_atomic_memory_migration(
File "src/powercontext/builtin/persistence/migrations/atomic_memory_v1.py", line 1380, in _verify_atomic_memory_migration
errors.extend(await residual_issues(connection, tables, thorough=thorough, collections_removed=collections_removed))
File "src/powercontext/builtin/persistence/migrations/atomic_memory_references.py", line 1298, in residual_issues
value = json.loads(bytes(row["payload"]))["value"]
KeyError: 'value'
The offline task is documented as resumable after an interruption, and it is, but every rerun fails at the same pre-removal acceptance step, so the deployment cannot complete the migration or start the new Server.
Expected behavior
residual_issues() judges parsed fields rather than assuming a payload shape, and does so for both stored layouts. A Source without the stored envelope is read through its top-level object, so it is either converted like any other Work Source or judged and reported; a Source that holds no Work payload still cannot hold a legacy Memory citation. Apply then completes, the collections leave the public tables, and verify returns ready: true.
Actual behavior
Apply aborts with KeyError: 'value' after importing all entries. The legacy collections stay in the public Artifact tables, so Server startup keeps failing the residual check.
Environment
- PowerContext version:
3b58d49d(feat(artifacts): share merge and restoration across families (feat(artifacts): share merge and restoration across families #1880)) - Database: SQLite, persistent file
- Python version: 3.12.13
- OS: macOS (arm64)
Are you willing to submit a PR to fix this bug?
- Yes, I would like to submit a PR.
Metadata
Metadata
Assignees
Labels
Type
Projects
- StatusShow more project fieldsDone
Activity