fix(recall-gate): score sufficiency on vector relevance, not fused rank

[delete-cloud]

Which issue or RFC does this PR close?

Closes #1916.

Updates RFC 1560 (#1560), implemented in #1596.

Rationale for this change

The recall sufficiency gate applies min_top_score and min_top_gap to normalized RRF values. For Memory these values
depend only on rank:

• a hit ranked first normalizes to 1.0, so min_top_score cannot fire;
• top_gap is a function of the admitted count, so the score check fails for 1–3 candidates and passes for 4 or more,
  whatever the evidence is.

Separately, when a later expansion round fails, prepare returns the round-zero candidates. This drops an expansion round
the gate has already assessed, even though RecallEffort still reports that round. The issue has a reproduction of both
defects.

The fix scores on the vector cosine relevance of the admitted vector channel. That is the scale the existing 0.3
admission baseline uses, and the scale min_top_score and min_top_gap describe. The fix also records an expansion
round only after the gate has assessed it.

What changes are included in this PR?

Summary

• Relevance signal. Memory and Topic Memory fusion attach relevance to each fused hit: the best cosine
  similarity among its admitted vector channels, clamp(1 - d² / 2, -1, 1) for unit L2 distance. FTS-only hits keep
  relevance=None. The field is declared exclude=True, like the existing retrieval_score and channel_scores,
  so it never enters model_dump() or any serialized projection. The fused score and ranking are unchanged.
• Gate scoring. top_score and top_gap come from relevance:
  • Each Scope's own first fused hit must have vector evidence before that Scope's hits contribute. Memory fuses once
    per Scope, so the verdict does not depend on the order in which Scopes are prepared.
  • The gap uses only scores at or above the round-zero semantic floor (0.3) and needs at least two of them.
  • A strong top relevance (>= 0.70) skips the gap check.
• weak-top-1 as a recorded reason, not a verdict. The score signals are read off the fused candidate window,
  which each family's search caps before the gate sees hits, so a strong vector-only hit can sit outside it until a
  lower lexical floor admits its second channel. A weak top therefore still expands while a family reports
  recoverable candidates; the reason is kept when the rounds are spent.
• Committed expansion. Each round's candidates are staged, budget-probed and assessed, and are committed only after
  the gate returns a verdict. A failure in a later round returns the last committed set. RecallEffort.signals records
  that set's aggregate signals. Committed candidates keep the latest round's fused order inside each family's cap, so
  a hit that only entered the window under a lowered floor is delivered to the Builder rather than dropped behind the
  round-zero prefix.
• Policy version. The policy identifier changes from powercontext.recall-gate.v1 to powercontext.recall-gate.v2.
• RFC 1560 (en and zh). Updated to describe the relevance signal, the per-Scope first-hit rule, the check order and
  the failure semantics.

Change map

Module	Before	After	Description
Shared search helpers (artifacts/search.py)	Each fusion had a private unit-L2-to-cosine conversion	unit_l2_cosine_similarity is shared	Admission and relevance use one conversion
Memory fusion (artifacts/memory)	MemoryHit carried a fused rank score	MemoryHit.relevance carries the admitted vector channel's cosine	Gives the gate a semantic signal
Topic Memory fusion (artifacts/topic_memory)	TopicMemorySearchHit carried a fused rank score	TopicMemorySearchHit.relevance carries the best admitted topic_vector / detail_vector cosine	Same signal for the second scored family
Recall gate (runtime/recall_sufficiency.py)	Score signals came from normalized RRF; the gap was checked for every scored family; policy v1	Score signals come from relevance, per Scope; the gap is limited to the baseline floor; RecallCandidate.scope_id; RecallEffort.signals; policy v2	Makes min_top_score and min_top_gap measure relevance
Recall loop (runtime/application.py)	Recorded a round before assessing it; a failure returned the round-zero candidates	Stages, assesses and then commits each round; a failure returns the last committed set; committed candidates keep the latest round's fused order inside the family cap; passes Scope ids to the gate	Keeps the delivered content and the trace consistent, and keeps newly surfaced evidence deliverable

Architecture and behavior

Scenarios:

• S1: the score input is vector relevance.
• S2: the first-hit rule is applied per Scope.
• S3: weak-top-1 records the insufficient score signal; expansion still runs while evidence is recoverable,
  and the reason survives at-max-rounds rewriting. The >= 0.70 strong-match bound exempts the gap check.
• S4: a later-round failure keeps the last committed round.
• S5: when the gate is disabled or does not expand, the output is unchanged.

Before

sequenceDiagram
  participant L as Expansion loop
  participant S as Family searches
  participant G as RecallSufficiencyGate
  L->>G: assess(round 0): top_score = normalized RRF (about 1.0)
  G-->>L: weak-top-1 iff 1-3 candidates
  loop insufficient and recoverable and rounds left
    L->>S: search with lowered admission floor
    S-->>L: hits
    L->>L: merge and record round
    L->>G: assess(merged)
    G-->>L: verdict
  end
  alt search or assess raises
    L-->>L: return round-zero candidates
  else
    L-->>L: return merged candidates
  end

Loading

After

sequenceDiagram
  participant L as Expansion loop
  participant S as Family searches
  participant G as RecallSufficiencyGate
  L->>G: assess(round 0): top_score = vector cosine, per-Scope first hit
  G-->>L: verdict
  loop insufficient and recoverable and rounds left
    L->>S: search with lowered admission floor
    S-->>L: hits
    L->>L: stage merged candidates (capped)
    L->>G: assess(staged)
    G-->>L: verdict
    L->>L: commit staged set, record round and signals
  end
  alt search or assess raises
    L-->>L: return last committed set (expansion-failed)
  else
    L-->>L: return last committed set
  end

Loading

The changed edges:

1. The value that assess reads (S1, S2).
2. Recording the round, which moves after assess (S4).
3. The failure branch, which returns the committed set instead of the round-zero arguments (S4).

When the gate is disabled, _gated_recall_effort is not entered, so S5 is unchanged.

Are there any user-facing changes?

• HTTP API: none. PreparedContext keeps its four fields, and openapi/powercontext.yaml and the generated code
  are unchanged. search_response and topic_memory_search_response build their HTTP models field by field, so
  relevance is not serialized.
• In-process Python models: MemoryHit and TopicMemorySearchHit gain an optional relevance: float | None
  attribute (default None, exclude=True). It is readable on the in-process objects but absent from model_dump()
  and serialized output, matching the existing retrieval_score/channel_scores convention.
• Gate-enabled deployments:
  • Strong evidence no longer triggers expansion just because few candidates were admitted.
  • A weak top relevance is recorded as weak-top-1; expansion still runs while a family reports recoverable
    candidates, because the fused window can hide a strong hit that a lower floor surfaces.
  • A later-round failure delivers the last committed expansion instead of round zero.
  • RecallEffort.policy reports powercontext.recall-gate.v2.
• Gate disabled (the default): no behavior change.

Boundaries and risks

Invariant or boundary	Failure mode	Protection	Evidence or gap
Disabled gate or no expansion (S5)	Output changes when nothing expands	The loop is not entered when the gate is disabled; round zero is committed before any expansion	test_default_off_matches_a_sufficient_round_zero_byte_for_byte
FTS-only first hit (S2)	A lower-ranked vector hit gives an FTS-led Scope a weak score	A Scope's hits count only when its own first fused hit has relevance	test_fts_only_candidates_skip_relevance_thresholds, test_expansion_fts_only_hit_preserves_round_zero_scored_family
Scope order (S2)	The verdict depends on the order of referenced Scopes	Signals are grouped by RecallCandidate.scope_id; pooled max, mean and gap do not depend on order	Order-invariance and wiring unit tests; test_scoped_first_hit_rule_applies_per_scope_through_the_runtime
Lowered floor (S1)	Below-floor expansion hits manufacture a gap	The gap uses only scores at or above 0.3	test_lower_floor_candidates_cannot_manufacture_a_top_gap
Thin and weak round (S3)	weak-top-1 would mask thinness	thin-candidates and thin-families are checked before weak-top-1	test_gate_reports_thin_candidates_even_when_the_top_is_weak
Window-truncated strong hit (S3)	weak-top-1 stops expansion while a strong hit sits outside the capped fusion window	A weak top still expands while a family reports recoverable candidates; the reason is kept once rounds are spent; the committed set keeps the latest round's fused order so the surfaced hit is inside the delivery cap	test_bounded_expansion_surfaces_a_strong_hit_hidden_by_the_fusion_limit (verdict plus committed-set membership)
Later-round failure (S4)	The delivered content does not match the trace	Staged candidates are committed only after assess returns	test_later_round_failure_preserves_committed_expansion (search and assessment failure)
HTTP contract	relevance leaks into a response	HTTP mappers enumerate fields; no OpenAPI change	make contract-test
Threshold scale	0.35, 0.02 and the 0.70 strong-match bound are not calibrated for a production embedding model	Thresholds stay deployment configuration; 0.70 is a module constant	Not verified with a real embedding model. RFC 1560 open question 1 remains open
Cost accounting	A round that fails during probe or assessment is not counted in added_embeddings / added_generation_calls	Only committed rounds are counted, as RFC 1560 states	Differs from the search layer, which counts calls when issued

How was this change tested?

All commands were run at 67a024cd, rebased onto 968fbddc (the upstream TopicMemorySearchHit refactor from RFC
#1858 was merged: relevance is computed inside the new fuse_rrf pipeline and shares its exclude=True
convention):

make check                                                        # exit 0
                                                                  #   ruff check, ruff format, ty, prek hooks
make contract-test                                                # 49 passed
pytest tests/builtin/runtime/test_recall_sufficiency.py \
       tests/builtin/artifacts/memory/test_fusion.py \
       tests/builtin/artifacts/topic_memory/test_fusion.py        # 85 passed
pytest tests/e2e/test_recall_sufficiency_gate.py                  # 24 passed
pytest tests/builtin/artifacts/ tests/builtin/runtime/ tests/e2e/ # 1531 passed, 119 skipped, 1 failed

The one failure is tests/builtin/runtime/test_model_usage_recorder.py::test_commit_lock_rolls_back_usage_and_restores_connection
(IndexError: tuple index out of range). That file passes in isolation (29 tests) and is untouched by this PR; the
same IndexError flake has been observed on the unmodified base in full-suite runs, so it is a pre-existing
order-dependent issue.

New and updated tests:

• Gate unit tests:
  • rank-saturated scores with weak cosine;
  • an FTS-only first hit;
  • the admission-floor gap;
  • Scope order invariance;
  • a thin-and-weak round still reports thin-candidates and expands;
  • build_recall_candidates(memory_scope_ids=...) wiring, which makes the test fail if the call sites drop the Scope ids.
• Fusion tests: relevance on Memory and Topic Memory hits.
• End-to-end:
  • a SQLite prepare with a deterministic embedding at cosine 0.31 (weak-top-1, with one expansion recovering
    the floor-dropped hit);
  • a strong vector-only hit hidden outside the capped fusion window, surfaced by the lowered-floor round and
    asserted to reach the Builder's committed candidate set (the per-section render limit is a separate
    request-level display cap);
  • two Scopes joined by a context reference;
  • a later-round failure for both a search failure and an assessment failure.

The issue's reproduction script, run at both revisions:

Scenario	86537e4f (base)	67a024cd (this PR)
Cosine 0.90 / 0.50, one recoverable candidate	top_score=0.992, weak-top-1, at-max-rounds after 3 rounds	top_score=0.900, sufficient, 1 round
Cosine 0.31 / 0.31, one recoverable candidate	top_score=1.000, weak-top-1, at-max-rounds after 3 rounds	top_score=0.310, weak-top-1, 3 bounded rounds recovering floor-dropped evidence
Second expansion round raises	Delivers round-zero content	Delivers the first expansion round's content

Not verified:

• a real embedding model or the OceanBase backend; relevance has been exercised only with deterministic embeddings over
  SQLite;
• Topic Memory relevance through an end-to-end prepare (covered at the fusion level);
• tox across Python 3.12–3.14, and the full make test suite.

AI usage statement

Implemented with Devin (SWE-2). Reviewed with Cursor (Grok 4.7 and Claude Opus 5.5), Codex (GPT-6 Astra) and Devin
(SWE-2-Max).

[CLAassistant]

All committers have signed the CLA.

[Copilot]

🟡 Changes recommended

Terminal weak-score handling and stale relevance during expansion can produce incorrect gate decisions.

3 open findings

• Duplicate hits lose newly admitted channel relevance · New
• Weak top score can improve after bounded expansion · New
• Lexical insufficiency cannot trigger expansion after weak-top termination · New

What changed in this PR

Fixes recall sufficiency scoring by using vector cosine relevance and preserving committed expansion results.

Changes:

• Adds vector relevance to Memory and Topic Memory fusion.
• Updates gate scoring, scope handling, and expansion commit behavior.
• Revises tests and RFC documentation.

File	Description
src/​powercontext/​builtin/​artifacts/​search.py	Adds shared cosine conversion.
src/​powercontext/​builtin/​artifacts/​memory/​models.py	Adds in-process relevance.
src/​powercontext/​builtin/​artifacts/​memory/​fusion.py	Computes Memory relevance.
src/​powercontext/​builtin/​artifacts/​topic_memory/​models.py	Adds Topic Memory relevance.
src/​powercontext/​builtin/​artifacts/​topic_memory/​fusion.py	Computes Topic Memory relevance.
src/​powercontext/​builtin/​runtime/​recall_sufficiency.py	Scores gate signals using relevance.
src/​powercontext/​builtin/​runtime/​application.py	Stages and commits expansion rounds.
tests/​builtin/​artifacts/​memory/​test_fusion.py	Tests Memory relevance.
tests/​builtin/​artifacts/​topic_memory/​test_fusion.py	Tests Topic Memory relevance.
tests/​builtin/​runtime/​test_recall_sufficiency.py	Tests revised gate semantics.
tests/​e2e/​test_recall_sufficiency_gate.py	Tests runtime expansion behavior.
docs/​en/​rfcs/​1560_recall_sufficiency_gate.md	Updates English RFC.
docs/​zh/​rfcs/​1560_recall_sufficiency_gate.md	Updates Chinese RFC.

🧠 Review effort: Balanced

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[AlexStocks]

I walked through this and think the current behavior is the defensible one, not a loss: a candidate that gains an admitted vector channel only under the lowered floor always has cosine below the round-zero baseline (0.3), and _score_signals uses only at-or-above-baseline scores for the gap and can only lower top_score with below-baseline values. Promoting such a scope to scored would risk exactly the wrong outcome - converting a recoverable thin-candidates case into a terminal weak-top-1 on a 0.25 cosine. The RFC states this rule explicitly ("lower-ranked vector hits cannot turn an FTS-only first hit into a weak score"), and the accumulated candidate order keeps round 0 first, so the first-hit rule stays invariant to expansion order (there is an e2e test pinning exactly that). If the admitted-pool top cosine is ever surfaced to the gate (the separate fusion-limit gap), that would be the point to revisit this too.

[delete-cloud]

Agreed — this matches the intended invariant: a vector row that only survives the lowered floor is by construction below the 0.3 baseline, so promoting it to scored would invert a recoverable thin-candidates round into weak-top-1 on a below-threshold cosine. No change needed; the per-Scope first-hit rule and the ordering e2e test pin this.

[AlexStocks]

[P2] Confirmed, with the concrete mechanism: the gate's relevance signals are computed over the fusion-exposed hits only, not the admitted pool. MemoryService.search calls fuse_rankings(..., limit=coarse_limit) with coarse_limit = memory_limit (default 16, prepared_context.py:156), and _gated_recall_effort feeds exactly those fused hits into build_recall_candidates. Under RRF (k=60) any dual-channel hit (1/(60+r1)+1/(60+r2) ~ 0.032) outranks a vector-only hit (1/61 ~ 0.016) regardless of cosine, so once 16 dual-channel weak hits fill the fused top, a vector-admitted candidate with a much higher cosine stays invisible. Round 0 then reports top_score ~ 0.31-0.35, the verdict is weak-top-1, and the loop terminates - although the expansion round (lowered lexical floor admits the strong candidate into FTS, RRF lifts it into the top 16) could expose a 0.90 cosine.

So the RFC's justification - "Lowering the admission floor over the same retrieved pool cannot raise a top cosine that is already admitted" - holds for the admitted pool but not for the fusion-limited set the gate actually scores. The same applies to Topic Memory (topic_memory_candidate_limit = 8). Possible directions: (a) report the admitted pool's top cosine alongside the fused hits and score against that, so the terminal claim matches the gate's basis; (b) keep the terminal behavior but scope the RFC claim to the exposed set and document this crowding configuration as a known limitation; (c) treat weak-top-1 as terminal only when the exposed top equals the admitted pool's top cosine. Even without a code change, the RFC sentence needs the correction, since it is the stated rationale for ending expansion early.

[delete-cloud]

Addressed in d508f280: weak-top-1 is no longer loop-terminating — an insufficient score now takes the bounded expansion path like every other non-sufficient reason, and when the budget is spent with the top still weak the reason is kept rather than rewritten to at-max-rounds.

Among (a)/(b)/(c) this is the minimal variant: the gate keeps scoring the exposed window (not the admitted pool, which would score evidence that cannot be delivered), and the termination is removed rather than conditioned, because conditioning on "exposed top == pool top" (c) would need a new signal plumbed through fusion → search → gate for a bounded two-round saving. New e2e test_bounded_expansion_surfaces_a_strong_hit_hidden_by_the_fusion_limit reproduces exactly your construction (16 dual-channel weak hits + vector-rank-1 hit at 0.90 sharing one query term) through the real SQLite path and asserts the surfaced top_score≈0.90 and sufficient.

Residual: a surfaced hit informs the verdict but may still be dropped from delivery by _prefix_preserving_counts when the candidate cap is full — noted in the PR thread; whether delivery should re-rank by the new fused order is candidate selection (#1643).

[AlexStocks]

Confirmed with the check-order evidence: RecallSufficiencyGate.assess returns weak-top-1 before evaluating lexical_overlap, and _gated_recall_effort's loop condition treats weak-top-1 as terminal, so a candidate set that is both score-weak and lexically insufficient gets reason weak-top-1 and does not expand. Lexical insufficiency triggers expansion only when it is the deciding reason, i.e. the top passes both score checks. Suggest narrowing both the en and zh sentences to what the check order actually guarantees: candidate-count and family-coverage insufficiency expand (both are checked before weak-top-1); lexical insufficiency expands only when the top passes the score checks. The existing "a round that is both thin and weak still expands once" sentence is correct as written; it is the inclusion of lexical insufficiency in the same list that overstates.

[delete-cloud]

Fixed in d508f280 — the paragraph was rewritten in both locales: weak-top-1 is now described as a recorded insufficient-score reason, not a terminal verdict, with the fusion-window cap spelled out as the reason a weak top is still expandable. Your check-order point is resolved by the same change: since weak-top-1 no longer stops the loop, a score-weak + lexically-weak set still expands (its final reason remains weak-top-1), so the "may still trigger expansion" enumeration is accurate again.

[hidb4ai]

LGTM

[delete-cloud]

Thanks @hidb4ai for the review, and @Copilot for the findings — one of them identified a real defect this PR introduced, fixed in the latest push. Requesting re-review.

Disposition of the three Copilot findings

Finding 2 (weak top score can improve after bounded expansion) — confirmed, fixed. The fused window is capped (memory_candidate_limit, topic_memory_candidate_limit) before the gate sees hits, so "admitted but truncated" is a real state: a strong vector-only hit can sit outside the window until the lowered lexical floor admits its second channel and RRF moves it into view. I reproduced this with the real fuse_rankings: 16 dual-channel weak hits (cosine 0.31–0.34) fill the limit=16 window and hide a vector-rank-1 hit at cosine 0.90 whose text shares only one query term; after one lowered-floor round the hit enters the window and the exposed top relevance goes 0.34 → 0.90. weak-top-1 is therefore no longer a loop-terminating condition — it remains the recorded reason, and when the round budget is spent with the top still weak, the reason is kept instead of rewritten to at-max-rounds. New regression test: test_bounded_expansion_surfaces_a_strong_hit_hidden_by_the_fusion_limit (window full of weak dual-channel hits + one strong vector-only hit → expansion runs and surfaces it). RFC 1560 en/zh updated to describe the window-cap caveat.

Finding 1 (duplicate hits lose newly admitted relevance) — intentional, verified. Vector rows are admitted per row before fusion, so a round-0 FTS-only hit whose vector row first survives at the lowered floor necessarily has cosine < 0.3 — below min_top_score (0.35). Merging that metadata back would let below-floor cosines stamp weak-top-1 onto every Scope whose first hit is FTS-only, contradicting two invariants this PR pins (test_expansion_fts_only_hit_preserves_round_zero_scored_family). The builder also never reads relevance/matched_by, so the stale metadata cannot affect delivered content.

Finding 3 (lexical insufficiency vs weak-top-1 order) — wording already scoped, clarified further. assess checks weak-top-1 before weak-lexical, so a score-weak + lexically-weak round ends as weak-top-1 — which is what the paragraph's first sentence covers ("final reason is weak-top-1"). The "may still trigger expansion" clause refers to final reasons thin-*/weak-lexical. The paragraph was rewritten anyway as part of the Finding-2 fix.

Residual note

When a hidden strong hit is surfaced mid-expansion it counts toward the gate's score, but the per-scope delivery cap (_prefix_preserving_counts) still preserves the round-zero prefix, so the newly surfaced hit itself may not be delivered when the candidate cap is full. Whether delivery should re-rank by the new fused order is a candidate-selection question, tracked separately under #1643.

Verified on the pushed head: pytest tests/builtin/runtime/test_recall_sufficiency.py 52 passed; tests/e2e/test_recall_sufficiency_gate.py 24 passed; ruff/ty clean.

[delete-cloud]

Heads-up for reviewers: this fix is still being iterated on. Recent pushes are addressing review findings on weak-top-1 semantics and delivery ordering, and the branch may be force-pushed again. Please hold off on final review until I confirm the change is stable.

[delete-cloud]

Follow-up on the weak-top fix (67a024cd): an internal re-review surfaced one more consistency gap in the same area, now fixed on this branch.

Problem. When a hidden hit enters the fused window under a lowered floor, it was appended to the accumulated pool behind the round-zero prefix. The gate then scored it (sufficient on a 0.90 cosine) while _prefix_preserving_counts still dropped it at the family cap — the verdict improved but the delivered set did not contain the evidence.

Fix. Committed candidates now keep the latest round's fused order inside each family's cap (_latest_round_order); identities the latest round did not re-admit keep their accumulated order after the re-admitted prefix. The gate's scoring input still uses accumulated order, so the per-Scope first-hit rule and Scope-order invariance are unchanged — only the delivery ordering moved to the freshest ranking.

Verification. test_bounded_expansion_surfaces_a_strong_hit_hidden_by_the_fusion_limit now asserts the surfaced hit reaches the committed candidate set handed to the Builder (not just that the signal changed). Note the per-section render limit (assembly.sections[].limit, <= 8) is a separate request-level display cap: it can still clip how much of a committed set renders, which is unchanged behavior.

76 gate tests pass (52 unit + 24 e2e); ruff/ty clean.

[delete-cloud]

Update: the iteration is complete and the change is stable. All review findings have been addressed on 67a024cd (rebased onto current master):

• weak-top-1 is no longer loop-terminating — a weak top takes the bounded expansion path like other insufficient reasons, and the reason is preserved when rounds are spent.
• Committed candidates keep the latest round's fused order inside each family's cap, so evidence surfaced by a lowered floor is delivered to the Builder rather than trailing the round-zero prefix.
• RFC 1560 (en/zh) is consistent with the implemented semantics.

Ready for review. CI is pending maintainer approval to run; locally the full suite passes (1534 tests) with ruff/ty clean.