Claude Code Stop hook silently reindexes Milvus Lite on every turn

[M-Marbouh]

Summary

At current main (0078d04578480f51f0eafe25ffbb8f64cbfcb1b7), the Claude Code Stop hook is asynchronous and runs memsearch index after every captured turn, including when the configured backend is Milvus Lite.

The corresponding Codex hook only indexes from Stop for HTTP/TCP Milvus Server backends. The Claude hook has no backend guard, and its run_memsearch helper redirects stderr and converts every non-zero exit to success with || true. A failed per-turn index is therefore invisible to the user.

Current code

• plugins/claude-code/hooks/hooks.json:26-34: Stop runs with "async": true.
• plugins/claude-code/hooks/stop.sh:174-178: kills a prior index process, then unconditionally calls run_memsearch index "$MEMORY_DIR".
• plugins/claude-code/hooks/common.sh:126-132: suppresses stderr and ignores the command's exit status.
• For comparison, plugins/codex/hooks/stop.sh:197-202 guards the same operation to http* / tcp* URIs.

Observed failure

On Linux with memsearch 0.4.14, ONNX embeddings, and the default local Lite database:

• Markdown memory files continued to be written through July 16, while the affected collection's last segment write was a zero-byte data_003919_004001.parquet on July 3, with no later segments.
• Against a byte-identical throwaway copy of the corrupt collection, an explicitly scoped memsearch search ... --collection <collection> exited 1 during DescribeCollection: Parquet file size is 0 bytes. It did not return stale rows from the readable segments.
• A direct memsearch index <memory-dir> --force --collection <collection> also exited 1 on the same zero-byte segment. The failure was loud when invoked directly; the asynchronous Stop path hid it by suppressing stderr and converting the non-zero exit to success with || true.
• Before repair, the manifest sequence was 4,001 but only 3,428 physical rows were readable. Rebuilding from the explicit memory path into a clean collection produced 4,082 chunks, and an exact sentence from the July 16 file returned its own source.

Collection-scope note: bare memsearch search <query> uses the unrelated default collection memsearch_chunks. An earlier revision of this report incorrectly treated results from that default collection as evidence about the affected project collection; those observations have been removed.

This report does not assert that Lite file-lock contention caused the zero-byte segment. The confirmed problem is that the async Claude Stop path invokes a Lite index every turn and suppresses any failure, leaving a corrupt or unupdated derived index without a Stop-hook signal even though later direct operations fail loudly.

Expected behavior

The Lite indexing path should be serialized/single-flight (or limited to the existing SessionStart fallback), and a failed index should leave a detectable signal rather than being silently treated as success.

Related reports

• memsearch watch locks milvus.db, preventing search from working #80 addressed a persistent memsearch watch lock in Lite mode and intentionally retained Stop-hook indexing; it does not cover silent failure or overlap between asynchronous per-turn indexes.
• Windows: kill_orphaned_index can't reap native index processes (Git Bash) -> unbounded accumulation / OOM #600 concerns Windows/Git-Bash process reaping.
• Draft PR [codex] Fix Codex stop hook index fallback #599 changes the Codex Stop path, not the Claude Code path.

[askalot-io]

Additional data point for the same stop.sh:174-178 lines, on a failure mode this report doesn't cover: on a corpus large enough that one index run outlasts one turn, the kill-then-restart sequence means the index never completes at all while a session is active. It isn't only that a failure is silenced — the work is structurally starved.

kill_orphaned_index (common.sh:194) followed by run_memsearch index "$MEMORY_DIR" executes on every Stop. Each turn therefore terminates the in-flight index and starts a fresh full run from zero. Where an index takes longer than the user's turn cadence, that is an unbounded loop with no progress.

Observed on Linux, memsearch 0.4.16, ONNX embeddings, local Lite DB, 3.3 MB of markdown journals (66 files):

PID	Started	Outcome
1143982	11:02:00	killed, exit 144
1203800	11:13:28	killed
1217336	11:15:51	killed

Each start timestamp lands on a turn boundary. The collection's on-disk state never advanced past a partial WAL (collections/<name>/wal/wal_data_000001.arrow), and .index-state.json stayed pinned at "status": "running" because no run ever reached its completion write — which is also why the index_state_warning path in common.sh:120-145 reports "running for over 1h" rather than naming the restart loop.

Corroborating the timeline: the markdown journals were current through the same day, while the vector store had not been written since a date ten weeks earlier. Capture was healthy throughout; only the derived index was starved.

Confirming the mechanism rather than inferring it: re-running with an argv the sweep's pgrep -f "memsearch index $MEMORY_DIR" pattern cannot match — cd "$MEMORY_DIR" && memsearch index . -c <collection> — produced a process that survived turn boundaries and ran to completion. Nothing else changed. (The third sweep rule, pgrep -f "milvus_lite/lib/milvus", appears to be dead code against Lite 3.x, which runs the server in-thread; its only match here was the grepping shell's own command line.)

This strengthens the single-flight request in Expected behavior: an "already indexing → skip" guard fixes starvation and per-turn churn together, whereas making the failure merely non-silent would leave the loop in place. Worth noting the fix must be a skip, not a kill-and-restart with a longer interval — any restart policy reintroduces starvation for corpora above the size threshold.