Store: add reusable ranged-read snapshots

[zxpdemonio]

Description

Add a reusable ranged-read metadata snapshot for Store reads and use it in the Engram and reconstructed-shard read paths.

The motivation is to reduce TTFT-sized Engram read latency by avoiding repeated Store query/lease metadata work when the same key set is read repeatedly. The actual TransferEngine/RDMA data path is unchanged in this PR.

API and behavior

• Add PyClient::RangedReadSnapshot, a caller-owned snapshot of fresh batch_query() results.
• Add prepare_get_into_ranges_snapshot(keys) and refresh_get_into_ranges_snapshot(...).
• Add get_into_ranges_from_snapshot(...) for reads that must use the supplied snapshot and must not re-query or renew leases.
• Cache one snapshot per EngramStore layer and invalidate it on populate() and successful remove_from_store().
• Reuse the same snapshot abstraction in the parallel/reconstructed-shard read path when query results are already cached.

Snapshot refresh time is derived from the earliest lease timeout and is set to halfway through the remaining lease window. If any key fails query or has an expired lease, the snapshot is not treated as reusable. Ordinary get_into_ranges keeps the existing query-refresh behavior.

Module

• Mooncake Store (mooncake-store)
• Integration (mooncake-integration)

Type of Change

• Performance improvement
• Refactor
• API extension

How Has This Been Tested?

• Mooncake: 18 EngramStore tests passed, including Store-backed lookup, local lookup, invalid row ids, wrong shapes, missing tables, multi-layer isolation, populate/remove, and metadata access.
• Build passed for store, mooncake_master, and mooncake_client.
• Cross-machine RDMA benchmarks passed on two A10 + ERDMA machines.
• PR4083-style synthetic Engram read/write benchmarks passed with the same small-table shape and byte pattern used by the SGLang companion test.

Build:

cmake --build build-pr1-cached --target store mooncake_master mooncake_client -j 64

EngramStore regression:

MOONCAKE_BUILD_DIR=build-pr1-cached python3 scripts/test_engram_store.py

Result: 18 tests passed.

Checklist

• Changed code reviewed during development.
• Scoped build passed.
• Existing EngramStore regression tests passed.
• Cross-machine RDMA performance was measured against baseline.

AI Assistance Disclosure

• AI tools were used: OpenAI Codex assisted with implementation, review, testing, benchmark execution, and PR preparation.

Performance

Cross-Machine RDMA Engram Lookup

Two A10 + ERDMA machines; one host runs the writer/master and the other runs the reader. The benchmark uses the real Python store.EngramStore.lookup_into API.

Config:

• 8 heads
• 50,000 rows per head
• 64B row
• 1GB global/local segment
• 5 warmups, 7 measured iterations

Command shape:

ROWS=50000 HEADS=8 ROW_BYTES=64 ITERS=7 WARMUP=5 GLOBAL_SEGMENT_SIZE=1073741824 LOCAL_BUFFER_SIZE=1073741824 /root/run_engram_lookup_bench_variant.sh <variant> <master_port> <coord_port> 1 4 16 64 128 256

Results, median:

tokens	ranges	baseline	this PR	improvement
1	8	259.14 us/token	87.11 us/token	66.4% lower, 2.98x faster
4	32	65.30 us/token	24.41 us/token	62.6% lower, 2.68x faster
16	128	23.00 us/token	8.64 us/token	62.5% lower, 2.66x faster
64	512	11.58 us/token	7.81 us/token	32.5% lower, 1.48x faster
128	1024	9.50 us/token	6.95 us/token	26.8% lower, 1.37x faster
256	2048	8.47 us/token	6.52 us/token	23.0% lower, 1.30x faster

PR4083-Style Synthetic Engram Read/Write

Full DeepSeek-V4.1 serving was not repeated on this A10 setup. Instead, this benchmark repeats PR4083's synthetic Engram table layout and byte pattern with real cross-machine RDMA Store reads.

Shape:

• layer_ids=(1, 14)
• max_ngram_size=4, n_heads=8, head_dim=256, row_bytes=264, vocab_size=17
• 24 tables per layer
• Total written: 1,594,560 bytes
• Read batches: 1/4/8 tokens, 48 ranges per token across 2 layers
• 5 warmups, 30 measured iterations

Read results, median:

tokens	ranges	bytes/read	baseline	this PR	improvement
1	48	12,672	793.50 us/token	248.32 us/token	68.7% lower, 3.20x faster
4	192	50,688	217.67 us/token	77.14 us/token	64.6% lower, 2.82x faster
8	384	101,376	140.12 us/token	65.70 us/token	53.1% lower, 2.13x faster

Read results, p95:

tokens	baseline	this PR	improvement
1	820.70 us/token	258.24 us/token	68.5% lower, 3.18x faster
4	222.68 us/token	79.02 us/token	64.5% lower, 2.82x faster
8	148.38 us/token	70.93 us/token	52.2% lower, 2.09x faster

After the final cleanup, a short repeat run of the same PR4083-style benchmark still matched the PR numbers:

• 1 token: 248.83 us/token median
• 4 tokens: 78.19 us/token median
• 8 tokens: 66.12 us/token median