Skip to content

feat: local semantic search over cached abstracts #21

Closed

Description

@szaghi

Summary

Add mosaic search --local (or mosaic semantic QUERY) to perform embedding-based semantic search over the local SQLite cache without hitting the network.

Motivation

After accumulating hundreds of cached papers, users need a way to surface semantically similar results from their personal library — not just exact keyword matches. This is especially useful for re-discovering older cached papers that share a concept but not the same keywords.

Proposed behaviour

# Semantic search over local cache
mosaic search "self-supervised contrastive learning" --local

# Explicit semantic command
mosaic semantic "methods for learning representations without labels"

# Limit to papers with PDFs downloaded
mosaic semantic "attention mechanism" --downloaded-only

Implementation notes

  • Embedding model: Use a lightweight local model — sentence-transformers with all-MiniLM-L6-v2 (~22MB) is a good default, or fall back to TF-IDF cosine similarity if sentence-transformers is not installed
  • Index: Build an in-memory FAISS/numpy cosine index at query time from the abstracts column; for large caches (>10k papers), persist the index to a file alongside the SQLite DB
  • Extra dependency: sentence-transformers (optional; add as mosaic-search[semantic] extra)
  • Storage: Store embeddings as BLOB in a new embeddings table, keyed by paper_uid; regenerate lazily when an abstract is new or updated
  • Ranking: Return papers sorted by cosine similarity descending; display similarity score alongside the Rich table

Priority

Tier 3 — high value, higher complexity and dependency footprint

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions