Skip to content

Supersession demotion too weak: absolute penalty insufficient when old memory has higher raw cosine similarity #11

@Liorrr

Description

@Liorrr

Problem

The supersession demotion is absolute (supersedes_demotion = 0.15). When a superseded (OLD) memory has higher raw cosine similarity to a query than the NEW memory, the fixed 0.15 penalty is insufficient to flip the ranking.

Evidence — NR benchmark: 0/3 (0%)

NR-1 'Where does Sam work now?': Shopify/payments (OLD) still ranks #2 despite supersession edge to Stripe/billing (NEW). Old memory contains 'work at' literally → higher cosine sim for work-related queries.

NR-3 'What editor does Sam use?' PASSES because Neovim (shorter, direct) gets higher raw sim than VS Code + Copilot entry.

Root cause: score = raw_sim - 0.15 for superseded memories. If raw_sim_old - raw_sim_new > 0.15, the OLD memory still wins.

Fix Directions

  • Relative demotion: score *= 0.6 instead of score -= 0.15 (XS, recommended short-term)
  • Aggressive demotion: increase from 0.15 to 0.40 (XS)
  • Entity-boosted scoring at query time (M, KS73 entity system enables this)
  • KS75 store-time contradiction detection (L, real fix)

Discovered during KS73 overnight benchmark session (2026-04-08). Benchmark: benchmark_negative_recall.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions