MilvusStore misreports every local open failure as a Milvus Lite 3.x incompatibility — including transient lock contention — and advises moving the DB aside

[askalot-io]

Summary

MilvusStore.__init__ wraps every exception from MilvusClient(...) on a local URI in a fixed message asserting a Milvus Lite version incompatibility, and instructing the user to move the database aside and rebuild. The most common real cause on Lite — two processes opening the same data dir — produces that same message, so a transient lock conflict is reported as a permanent format problem, with destructive remediation advice attached.

Current code

memsearch/store.py:54-64 (0.4.16):

try:
    self._client = MilvusClient(**connect_kwargs)
except Exception as exc:
    if is_local:
        raise RuntimeError(
            "Failed to open the local Milvus Lite database. If this database was created "
            "with an older Milvus Lite release, it may not be compatible with Milvus Lite "
            "3.x. Move the existing .db file aside, then rebuild the index from your "
            "source markdown files with 'memsearch index'. Alternatively, use Milvus "
            "Server via Docker or Zilliz Cloud."
        ) from exc
    raise

except Exception with a single hardcoded diagnosis. The real cause is preserved in __cause__ but never inspected, and the CLI does not surface it.

Reproduction

Run two memsearch index processes against the same local DB. The second fails with the version-incompatibility text, while the chained cause is a lock error:

File ".../milvus_lite/db.py", line 932, in _acquire_lock
    fcntl.flock(fd, fcntl.LOCK_EX | fcntl.LOCK_NB)
BlockingIOError: [Errno 11] Resource temporarily unavailable

milvus_lite.exceptions.DataDirLockedError: another process holds the lock on
'/root/.memsearch/milvus.db': [Errno 11] Resource temporarily unavailable

pymilvus.exceptions.ConnectionConfigException: (code=1, message=Open local milvus failed)

RuntimeError: Failed to open the local Milvus Lite database. If this database was created
with an older Milvus Lite release, it may not be compatible with Milvus Lite 3.x. Move the
existing .db file aside, then rebuild the index ...

The database in that run had been created minutes earlier by Milvus Lite 3.x itself, so the asserted diagnosis was impossible on its face.

Why this matters more than a wording nit

1. The advice is destructive and the trigger is transient. "Move the existing .db file aside, then rebuild" discards a working index to resolve a lock that clears on its own. For a large corpus the rebuild is expensive; for a user without the source markdown it is unrecoverable.
2. Concurrent opens are routine, not exceptional. Per Claude Code Stop hook silently reindexes Milvus Lite on every turn #629 the Claude Code Stop hook launches memsearch index on every turn after killing the previous one. Overlap between a hook-spawned index and any manual memsearch search / index / stats is expected, so this message is on a hot path.
3. It actively misdirects diagnosis. Encountering it, I moved a genuinely incompatible 2.x database aside — correct in that instance — and then hit the identical message against the fresh 3.x store. Only reading __cause__ revealed the second failure was a lock, not a format. Anyone who trusts the message will keep rebuilding an index that was never broken.

Note that the two causes are trivially distinguishable at the point of the catch: a genuine 2.x database is a single file, whereas 3.x uses a directory (collections/, databases/, LOCK).

Suggested fix

Inspect the cause before asserting one. Roughly:

• DataDirLockedError (or a BlockingIOError in the chain) → "another process is using this database (an index may be running); retry shortly", no rebuild advice, ideally a distinct exception type so callers can retry rather than surface a hard error.
• Path exists and is a file rather than a directory → the current incompatibility message, which is then accurate.
• Anything else → re-raise, or report generically with the underlying error included rather than substituting a guess.

At minimum, include exc in the message so the real cause is visible without a traceback, and drop the "move the .db file aside" instruction from any branch that has not established the database is actually unreadable.

Environment

memsearch 0.4.16 (uv tool install "memsearch[onnx]"), Linux, Python 3.12, local Lite backend, ONNX embeddings.