Commit caeab94
feat(agent_loop): add system-prompt guidance for relative time + multi-source asks
v0.18-α1-2 KRRA Conv diagnostic identified two prompt-shaped Qwen3.5-27B
failure patterns on conversational Korean queries:
1. "올해 예산은?" / "내년도 일정은?" — agent injected literal year
("2024"/"2025") into deep_search, throwing away corpus docs from
other years. Now told to search the topic first and only narrow by
year if the topic search returns too many candidates AND there's
evidence the user wants a specific year.
2. "X 관련 자료 있어?" / "내용 보여줘" — agent terminated after 1
deep_search returned 1-2 docs and answered "yes, here is one source".
Now told that "자료/내용/정보" asks for multiple sources, do at
least one paraphrase pass before concluding.
Both lines are general-purpose, not KRRA-specific, so they apply to any
Korean / English corpus.
The KRRA Conv -23pp regression vs v0.13 GPT-4o-mini (47% vs 70%) is
documented as a known issue: it stems from a recall ceiling on broad
topical queries (예산 / 인권 / 자료) where 5 ground-truth docs share a
vague topic word and deep_search top-K is dominated by phrase nodes.
Verified by direct corpus probe — 0/150 GT IDs reachable as direct node
ids; matching works only via properties.doc_id and depends on the right
docs landing in deep_search top-K. Real fix requires higher recall or
reranker-on-by-default; deferred to v0.19+ track.
45/45 agent + calibration tests pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>1 parent fa82a66 commit caeab94
2 files changed
Lines changed: 31 additions & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
6 | 6 | | |
7 | 7 | | |
8 | 8 | | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
9 | 29 | | |
10 | 30 | | |
11 | 31 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
101 | 101 | | |
102 | 102 | | |
103 | 103 | | |
104 | | - | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
105 | 115 | | |
106 | 116 | | |
107 | 117 | | |
| |||
0 commit comments