Retrieval fixes behind default-off flags: vector type tagging, input types, Cohere rerank, score ordering

[StockerMC]

Merging this is a no-op until a flag is set

Every behaviour change is behind a feature flag that defaults to off. With no new
environment variables set, the deployed system serves the same responses, in the
same order, from the same candidate pool as it does today. The flags exist so the
new path can be turned on locally and evaluated first.

Flag	Default	Effect when on
RETRIEVAL_RERANK_ENABLED	off	Ordering only. Wider candidate pool, creator-video filter that also matches legacy vectors, Cohere rerank, results ordered by rerank score. Same response keys either way.
CREATORS_API_EMIT_MATCHES	off	Response shape only, frontend route. Adds the matches array (entries carrying a nested creator_videos) that the reels view reads. No reordering.
RETRIEVAL_DOC_INPUT_TYPE_ENABLED	off	Indexing path: corpus content embedded with input_type=search_document
COHERE_RERANK_MODEL	rerank-v3.5	Rerank model (only read when reranking)
RETRIEVAL_CANDIDATE_POOL	30	Candidates pulled from Pinecone before reranking
RETRIEVAL_TOP_N	10	Results kept after a successful rerank

Flag helpers: backend/utils/feature_flags.py, frontend/src/lib/featureFlags.ts.
Documented in the README under Retrieval feature flags.

What changed

1. Dead candidate filter. backend/API.py filtered vector matches on
metadata["type"] == "creator_video", but backend/background_worker.py — the
only path that actually indexes videos — never wrote that key. Only
generate_embeddings_job (backend/jobs/tasks.py:189) did, and nothing enqueues
it, so vector_matches was always empty.

• backend/background_worker.py:432 now writes "type": "creator_video", and
  backend/utils/vectordb.py:120 writes "type": "product". Products and videos
  share one index and namespace, so the key is the reliable discriminator.
• is_creator_video_match() (backend/utils/vectordb.py,
  isCreatorVideoMatch() in frontend/src/lib/vectordb.ts) treats a vector as a
  creator video when type == "creator_video", or when type is absent and
  video_id is present.
• Writing the key is unconditional and forward-looking. Consuming it is not: with
  the flag off, backend/API.py keeps the original strict filter, so the endpoint
  keeps returning an empty vector_matches exactly as it does now.

No backfill is required. The video_id fallback covers every vector indexed
before the key existed.

2. Embedding input types. text_to_embedding used input_type="search_query"
for everything. embed-english-v3.0 is asymmetric. Corpus writers
(embed_products, background_worker.py:415, jobs/tasks.py:180) now call
document_to_embedding(); live query paths are unchanged.

Enabling RETRIEVAL_DOC_INPUT_TYPE_ENABLED requires a full re-embed of the
Pinecone index. Everything currently in the index was written as
search_query. Turning the flag on without re-embedding leaves a mixed index
whose vectors are not comparable, which degrades retrieval rather than improving
it. That is why it is a separate flag from the serving one and why it defaults
off.

3. Cohere Rerank stage. backend/utils/rerank.py and
frontend/src/lib/rerank.ts. Retrieves RETRIEVAL_CANDIDATE_POOL (30) candidates
and keeps RETRIEVAL_TOP_N (10). Every failure mode — missing key, client
construction, API error, empty result, out-of-range index — returns null so the
caller keeps similarity order instead of failing the request. The Cohere client is
built once and cached against the key it was built from, not per request.

4. Serving path and ordering. Ranking lives in backend/utils/creator_ranking.py
and frontend/src/lib/creatorRanking.ts, imported by both endpoints. Pre-computed
matches (scored 0-10 by the keyword scorer in backend/utils/relevance.py) and
vector matches (Pinecone cosine similarity) are deduplicated by video_id and
scored in one rerank pass. Their native scores are on unrelated scales and
cannot be merged directly; one pass puts every candidate on the same 0-1 scale.

The Cohere score is written to rerank_score. The stored relevance_score is
never overwritten, because utils/relevance.py compares it against a 0-10
threshold. Both endpoints agree on what each key means.

If rerank is unavailable, the fallback is pre-computed sorted by relevance_score
(falling back to similarity_score), then vector sorted by Pinecone score. The
caller's limit applies on that path and nothing is dropped; truncation to
RETRIEVAL_TOP_N happens only when rerank succeeds.

5. backend/scripts/eval_rerank.py. Prints a product's top N in similarity
order next to rerank order with scores, and how many positions moved. It calls the
rerank stage directly, so neither flag needs to be set. Read-only. I did not run it.

cd backend
python scripts/eval_rerank.py --list
python scripts/eval_rerank.py --product-id <uuid>

Review fixes in this revision

The rerank flag was not an ordering flag on the frontend. reels/page.tsx:116
reads result.matches, which the Next.js route has never returned, so the
product-scoped reels view renders empty today. Emitting matches under
RETRIEVAL_RERANK_ENABLED meant flipping that flag turned the view on from empty
rather than reordering it, and the flag's documented effect was wrong.

Resolved by splitting it: matches now sits behind its own
CREATORS_API_EMIT_MATCHES, which changes the response shape and nothing else.
RETRIEVAL_RERANK_ENABLED is ordering only. The two are independent — the UI fix
can ship without the ranking change, either can be reverted alone, and neither
flag's name overstates what it does. Emitting matches unconditionally would have
been the smaller diff, but that changes a live response with every flag off, which
is the one thing this PR promises not to do.

Rows whose creator_videos embed missed are dropped from matches rather than
emitted as null, which the reels page then dereferenced; that page guards the
field as well.

match_score is not a column. Both fallback sorts used
match_score, which belongs to the legacy product_matches table — never present
on product_creator_matches rows, so every key evaluated to 0 and the "fallback to
similarity order" was a stable no-op preserving created_at DESC. Both now sort on
relevance_score / similarity_score, the fields background_worker.py actually
writes.

The rerank path clobbered relevance_score. {**row, "relevance_score": score}
overwrote the stored 0-10 keyword score with Cohere's 0-1 score, so a row scored 8.5
came back as 0.31 and failed the min_score = 4.0 comparison in relevance.py:97.
Cohere's score goes to rerank_score now.

The failure path silently dropped results. ordered[:keep] with
keep = min(limit, top_n) meant a rerank failure with 12 pre-computed rows and
?limit=50 returned 10 matches and an emptied vector_matches — worse than the
flag-off response. The fallback now respects the caller's limit and returns both
lists whole.

Feeding that path, search_text was initialised to "" and only assigned inside
the pinecone_id branch, so a product that was never indexed always reached
rerank_documents("", ...), which short-circuits to None and took the truncating
fallback every time. The rerank query is now built from whatever product fields
exist, separately from the text vector search embeds (which is unchanged).

A fresh cohere.ClientV2 per request, never closed. cohere 5.20 gives each
instance its own httpx.Client, so every request leaked a connection pool. The
client is cached at module level, keyed on the API key so rotation and removal
still take effect, and construction moved inside the try — an ImportError or a
bad CO_API_URL now returns None and falls back instead of raising a 500, which is
what the module docstring always claimed. reset_client_cache() exists for tests;
an injected client still bypasses the cache entirely.

Order-dependent test fixture. The vectordb fixture imported utils.vectordb
inside a patch() context, which only patches when the module is not already in
sys.modules. It now reloads inside that context.

That stale client turned out to be the smaller half of the problem:
sync_shopify_products_job imports utils.vectordb from inside its body, and
pc.Index(name) resolves the index host through a live call to api.pinecone.io
with whatever key .env holds. conftest.py now stubs both client constructors for
the session and pins dummy keys, so a test run cannot reach Pinecone or Cohere
regardless of which files are selected.

Tests that can actually fail. The Cohere fake was a bare MagicMock, so
renaming documents= to docs= would still have passed, and
rerank_creator_candidates had no coverage at all. The fake is now autospecced from
the real cohere.ClientV2, so a wrong call shape raises TypeError, and the kwargs
are asserted key by key. One test pins that ClientV2.rerank still resolves to
V2Client.rerank through the ClientV2(V2Client, Client) MRO — requirements.txt
pins cohere>=5.18,<6 and pinecone>=8,<9 for the same reason, since the v1
rerank signature is incompatible.

Each new assertion was checked by mutating the source: renaming a Cohere kwarg,
writing the Cohere score to relevance_score, truncating the fallback to top_n,
sorting on match_score, and emitting rows with a null video all fail the suite.

Ranking moved out of API.py so it could be tested at all — importing API.py
runs load_dotenv() against real credentials and constructs live clients.

Behaviour when the flags are on

• RETRIEVAL_RERANK_ENABLED: on a successful rerank the served list is capped
  at min(limit, RETRIEVAL_TOP_N). The UI sends no limit, so it goes from up to
  50 date-ordered rows to 10 rerank-ordered ones. Raise RETRIEVAL_TOP_N if that
  is too aggressive. On failure the list is not capped.
• CREATORS_API_EMIT_MATCHES: adds one key. The reels view starts rendering; order
  is whatever the rerank flag decides.

Verified vs unverified

Verified by static checks and offline tests only:

• python3 -m pytest tests/test_retrieval.py — 68 passed (was 31).
• python3 -m pytest tests/test_job_queue.py tests/test_retrieval.py — 46 passed,
  in both file orders. Previously 2 failed.
• python3 -m pytest tests/test_retrieval.py tests/test_job_queue.py tests/test_metrics.py tests/test_redis_client.py tests/test_shopify_oauth.py —
  144 passed. (test_real_search.py, test_youtube.py, test_discovery.py,
  test_reels_fetch.py and test_improved_discovery.py call live APIs and were
  deliberately not run.)
• npm test in frontend/ — 20 passed. Offline, node --test, no new
  dependencies.
• tsc --noEmit and eslint clean on the changed frontend files.
• Rerank call shapes read off the installed SDKs: cohere==5.20.0
ClientV2.rerank(*, model, query, documents, top_n) resolving to V2Client, and
  cohere-ai@7.20.0 rerank({model, query, documents, topN}).

Flag-off is still a no-op. The backend keeps the exact search_text expression it
had, and rank_creator_candidates is only reached inside if rerank_enabled. The
frontend returns the same keys in the same order with the same values. One
disclosed exception: pre-computed entries in the creators array now also carry
relevance_score and similarity_score, two real columns that were being read for
sorting anyway. Nothing consumes creators — the only caller of this route is
reels/page.tsx, which reads matches.

Unverified — nothing was executed against Cohere, Pinecone, Supabase, YouTube, or
Gemini, and the backend and worker were never started:

• No live request has gone through the rerank stage. The API contract is taken from
  the installed SDK type definitions, not from a response.
• rerank-v3.5 is not confirmed to be enabled on this Cohere account.
• No claim about retrieval quality is backed by evidence. Whether reranking
  produces better matches than the current ordering is exactly what
  eval_rerank.py is there to answer.
• The end-to-end flag-on path (Pinecone pool → filter → rerank → ordered response)
  has not been exercised against real data.

Ambiguity flagged rather than guessed

• Scoring both sources in one rerank pass is a judgment call; a weighted blend that
  preserves the keyword scorer's signal is a defensible alternative.
• RETRIEVAL_TOP_N=10 shrinks the served list on the success path. Reasonable for a
  swipe feed, worth a second opinion.
• CREATORS_API_EMIT_MATCHES is a temporary flag. Once the reels view is confirmed
  working in production it should be deleted and matches emitted unconditionally,
  or the page corrected to read creators/video and the key dropped entirely.
• frontend/.env.example gained PINECONE_API_KEY, PINECONE_INDEX_NAME, and
  COHERE_API_KEY. The route already required them; they were simply undocumented.

The transcript pipeline is untouched.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
maatchaa	Ready	Preview	Aug 26, 2026 12:30am