feat: baked pubkey→index map (mapgen + release bake) — no cold sweep#9
Merged
Conversation
Adds the baked account-index map so /account?pubkey= resolves instantly, without the multi-minute cold-start epoch-ledger sweep. - index_map: compact [covered][16-byte blake2b key][u32 index] format (binary-searchable; mina-verify-mobile-compatible), with unit tests. - lib: factor the chunked sweep into mina_light_node::sweep_index_map (shared by the server's tail-sweep and the generator). - mina-light-node-mapgen: release-time generator — fetch+verify tip, derive the staking-epoch-ledger root, sweep, write <network>.bin. - server: load LIGHT_NODE_INDEX_MAP at startup (empty placeholder => sweep fallback); index keyed by addr-hash so baked + swept entries unify. - deploy: image bakes deploy/index_map.bin (empty in-tree placeholder) at /opt and sets LIGHT_NODE_INDEX_MAP. - release.yml: index-map job runs mapgen per network, bakes the map into each image, and attaches the .bin to the release (placeholder fallback if a sweep is flaky, so it never blocks the image). Generation is an RPC sweep, not the S3 RocksDB tarball: the tarball stores the leaf index in Mina's internal RocksDB Location encoding (unproven to decode; the mobile app swept too) — kept as a follow-up. Validated live on devnet: mapgen wrote a 91585-pair / 1.6 MB map; the server loaded it (81611 keys) and answered /account?pubkey= instantly on the first verified tip (h529032), no cold sweep. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0117RNu925HJ97mFHLbQG4Y9
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Removes the multi-minute cold-start sweep:
/account?pubkey=resolves instantly from a bakedpubkey → leaf-indexmap.What
index_map— compact[u64 covered][16-byte blake2b(addr)][u32 index]blob, binary-searchable, format-compatible with the mina-verify-mobile embedded map. Unit-tested (build/lookup/load roundtrip).mina-light-node-mapgen— release-time generator: fetch + verify the tip → staking-epoch-ledger root → sweep → write<network>_index_map.bin.LIGHT_NODE_INDEX_MAPat startup → by-pubkey is instant; the background sweep then only fills the appended tail. Empty placeholder ⇒ runtime sweep (unchanged behaviour).deploy/index_map.bin(an empty in-tree placeholder) at/optand sets the env;release.ymlrunsmapgenper network, bakes the real map into each image, and attaches the.binto the GitHub Release. A flaky sweep falls back to the placeholder, so it never blocks the image.mina_light_node::sweep_index_map(shared by the server + mapgen).On the S3 tarball
The map is generated by an RPC sweep, not the
s3://snark-keys-ro.o1test.net/epoch_ledger_<hash>.tar.gztarball. That tarball is a RocksDB database — the public key is readable in the key, but the leaf index lives in the value as Mina's internalLocationencoding, which is unproven to decode in Rust (the mobile app swept rather than decode it). Decoding the tarball is kept as a follow-up ([ ]in the README); the RPC sweep achieves the same baked artifact today, reusing validated code.Validated live (devnet)
fmt + clippy
-D warningsclean;cargo testgreen (3 index_map + 8 relay unit, 2 gated e2e).🤖 Generated with Claude Code