Skip to content

Embedding materialization hangs on 23K+ chunks during build #106

Description

@marklubin

Problem

fastembed hangs during _materialize_search_surface() when processing ~23,797 reference chunks with modes=["fulltext", "semantic"]. Process shows 0% CPU — deadlock or blocked I/O.

Investigation so far

  • fastembed standalone: 23K docs in 70s (works fine)
  • fastembed from asyncio executor thread: 12s (works fine)
  • The hang is in the synix indexer path, not fastembed itself
  • FTS5-only materialization for same chunk count works (takes ~16 min)

Environment

salinas: AMD CPU, 120GB RAM, Python 3.13.12, fastembed 0.8.0, onnxruntime 1.24.4

Current workaround

Reference surface uses modes=["fulltext"] only. Main surface (7 artifacts) has semantic enabled.

Next steps

  • Add logging inside _generate_embeddings and _materialize_search_surface
  • Test with actual build path at increasing chunk counts (100, 1K, 5K, 10K, 23K)
  • Check if the hang is in SQLite write after embeddings, not the embedding generation
  • Check ONNX thread settings (OMP_NUM_THREADS=1)

See docs/issues/fastembed-hang-23k-chunks.md

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions