[ThomasWaldmann]

The index/ chunk index fragments, cache/checked-packs and cache/referenced-by-archive.* are now stored in the repository key's envelope, exactly like the metadata and data slots of the objects in the packs: encrypted and authenticated in the encrypting modes, authenticated only in the authenticated-* modes. No new crypto, no mode-specific code in the repository layer.

Fixes #9819, fixes #10235.

Changes

• Repository.set_key(), store_encrypt_store(), store_load_decrypt(): wrap a store object in key.encrypt(b"", value, aad=...). The AAD is b"borg-store-object\0", the repository id, a tag and the object name (tag n) or, for the content-addressed index fragments, the namespace index (tag h). An object copied to another name, or from another repository using the same key material, fails the authentication. Without a key, both fail closed with the new Repository.KeyRequired (rc 28), never falling back to plaintext.
• The repository key is set wherever a key is loaded or created (key_factory, key_creator, Manifest), including the source repository of borg transfer.
• borg repo-create writes the empty chunk index after creating the key (Repository.create() no longer does).
• index fragments:
  • A fragment's name is the store hash of the stored envelope, so borg check keeps verifying index/ by name (server-side for rest://) without the key. Loading a fragment does not hash it to verify its name: the authentication already proves its content.
  • The envelope is randomized in the encrypting modes, so a fragment name no longer tells whether its content is already stored. The client remembers the plaintext hash of the fragments it read in the session and does not store a fragment with known content again (unless forced), so e.g. a repack that crashed between storing and deleting still converges.
• A corrupt chunk index (a fragment that fails the authentication or does not deserialize) aborts every command that needs it, with the new CorruptChunkIndexFragment error (rc 94): Run "borg check --repair" to rebuild the chunk index. Only the commands that rewrite the whole index anyway, under an exclusive lock, rebuild it from the packs instead: borg check --repair, borg compact and borg repo-compress (borg compact --dry-run aborts). Normal commands never modify index/ because of a corrupt fragment.
• borg check: the repository check reports such a fragment as an index error (rc 1 with --repository-only). The archives check needs the index, so a full check without --repair then aborts (rc 94). --repair rebuilds the index from the packs and drops the fragment.
• cache/checked-packs and the archive reference caches: envelope instead of an appended store hash. A reference cache copied to another archive's name is ignored and rebuilt by scanning that archive (a hostile store can no longer make compact drop chunks an archive uses).
• borg debug get-obj / put-obj / delete-obj load the key via with_repository() now (they access the chunk index), including the usual security checks; delete-obj without a key aborts instead of keeping the superseded gap bytes.
• PackTracker takes the repository instead of its store.
• docs: new "store object envelope" section in data-structures.rst, updated packs.rst, security.rst (attack model, what the store sees), frontends.rst (new errors), changes.rst and the borg check help.

What stays visible to the store

Object names: the number and sizes of the index fragments (a rough chunk count), the archive ids that have a reference cache, that a check ran. cache/chunkindex-invalid (an empty marker) and locks/* stay plain.

Existing beta repositories

No migration code (b25 already requires new repositories). On an older repo, the plaintext fragments fail the authentication, so the commands that need the chunk index abort with CorruptChunkIndexFragment; borg check --repair (or borg compact) rebuilds the index from the packs. Covered by test_plaintext_index_fragment_of_an_older_beta.

Tests

• Unit tests per mode (AES-OCB, ChaCha20-Poly1305, authenticated-sha256/blake3): roundtrip, tampering / renamed / copied objects rejected (also across repositories, and between an object name and a namespace), payload visibility, KeyRequired, plaintext fragments of older betas, known fragments not stored again, abort on a corrupt fragment vs. rebuild when the index is rewritten anyway, check/repair of fragments failing the authentication.
• Archiver tests (store_objects_test.py): end-to-end in aes256-ocb and authenticated-sha256 with inspection of the raw store objects, a copied reference cache, the older-beta plaintext fragment (commands abort, borg check --repair fixes it), and borg compact rebuilding a corrupt index (--dry-run aborts).
• Test infrastructure: repositories opened by the tests get a key (open_repository() loads the real key, repository_test.py / cache_test.py use a test key).
• Full suite passes locally on macOS (mount tests excluded, they are broken locally since macOS 27). The abort and rebuild paths were also run on the CLI with their real exit codes.

🤖 Generated with Claude Code

[codecov]

Codecov Report

❌ Patch coverage is 96.52174% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 88.33%. Comparing base (d576bb4) to head (cc10f61).
⚠️ Report is 5 commits behind head on master.
✅ All tests successful. No failed tests found.

Files with missing lines	Patch %	Lines
src/borg/crypto/key.py	85.71%	1 Missing and 1 partial ⚠️
src/borg/cache.py	97.22%	0 Missing and 1 partial ⚠️
src/borg/repository.py	98.11%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master   #10406      +/-   ##
==========================================
+ Coverage   88.28%   88.33%   +0.04%     
==========================================
  Files         103      103              
  Lines       18904    18939      +35     
  Branches     2937     2938       +1     
==========================================
+ Hits        16690    16729      +39     
+ Misses       1540     1539       -1     
+ Partials      674      671       -3     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

[mr-raj12]

After #10379 merged, the test_repair_finish_* tests open a keyless Repository and now hit KeyRequired via repository.chunks → store_load_decrypt (7 failures on #10412's CI) — they will need a set_key once this branch is rebased.