fix(email): stop the assistant from narrating what the turn's tools don't support (#2659)

The email agent used to say things its own tools hadn't earned: a
confident "archived" or "starred" with zero tool calls that turn, a
pre-scan claiming "no urgent or actionable items" while its own scan
result held some, unread counts described as spanning "across your
connected mailboxes" when the number is per-INBOX-scoped, and internal
payload scaffolding — a `[shown to the user]` marker,
`[suggested_archives]`-style field names, raw provider ids, undecoded
`\uXXXX` escapes — leaking into replies instead of being summarized. It
now either backs a claim with a matching tool call from that same turn,
or replaces the claim with an honest, data-grounded fallback;
`check_followups` also returns an explicit `count` so a long list has
nothing left to miscount.

Root cause for the unicode leak: every `json.dumps` call in the shared
agent loop (`src/gaia/agents/base/agent.py`) that builds model-visible
tool-result text was missing `ensure_ascii=False`, so non-ASCII
characters escaped as literal `\uXXXX` text — and could inflate a
payload's measured length enough to trigger truncation it didn't need.
Fixed at every call site there, which benefits every agent built on the
shared loop, not just email; the blast-radius check (`tests/unit/ -k
"agent_loop or tool_result or truncat"`) is green.

This PR also now folds in #2656's calendar-conflict grounding (merged to
main after this branch), which independently added the same kind of
check — a post-hoc verdict the model narrated without actually calling
`detect_calendar_conflicts` — but wired as a second, separate pass over
the final answer. Left as two passes, one turn could get two independent
disclaimers appended to a single reply, so the calendar check now lives
in this PR's shared `answer_grounding.py` framework as one more rule,
behind the same single hook.

## Test plan

- [x] `PYTHON_KEYRING_BACKEND=keyring.backends.null.Keyring
.venv/bin/python -m pytest tests/unit/agents/email/ tests/unit/email/
hub/agents/email/python/tests/ -q` — 2414 passed
- [x] `PYTHON_KEYRING_BACKEND=keyring.backends.null.Keyring
.venv/bin/python -m pytest tests/unit/ -q -k "agent_loop or tool_result
or truncat"` — 100 passed, 0 failed (proves the shared-loop unicode fix
doesn't regress other agents)
- [x] `go build ./... && go test ./...` in `tui/` — all packages green
- [x] `python util/lint.py --all` — all checks pass (MyPy warnings
pre-existing, non-blocking)
- [x] Every guard function mutation-tested, including the newly-folded
calendar-conflict rule: revert the fix alone, confirm its test fails,
restore, confirm green again
- [x] Dedicated composition test proves the calendar-conflict and
attention-card checks — both append-only and independent — fire together
on the same turn rather than one short-circuiting the other
- [ ] Dispatcher: 3 fresh-process `check_followups` runs against a
22-item mailbox (#2622 AC2), a 10+-turn TUI session reproducing the
empty-tool-trace narration (#2621 AC1), both capturing the trailing "N
steps · N tools" line

Closes #2621
Closes #2622
Closes #2636
Closes #2637

Author: itomek

hub/agents/email/python/CHANGELOG.md

@@ -103,6 +103,55 @@ contract version is tracked separately as
 
 ### Fixed
 
+- **The assistant no longer narrates things the current turn's own tools
+  don't support (#2621, #2622, #2636, #2637).** Four related honesty
+  defects, all guarded by one new mechanism: a mutation ("archived",
+  "starred", "marked read", "moved to Trash", ...) was sometimes narrated
+  as done with zero tool calls in that turn (observed 7 times in one long
+  session, correlating with conversation length); `check_followups`
+  reported fewer awaiting-reply items than its own intact result actually
+  held, dropping a different subset on each of 3 fresh runs; a pre-scan's
+  framing sentence could claim "no urgent or actionable items" while its
+  own scan result carried non-empty urgent/actionable lists, or describe
+  a per-INBOX-scoped `total_unread` as spanning "across your connected
+  mailboxes"; and internal render/envelope scaffolding — a
+  `[shown to the user]` context marker, `[suggested_archives]`-style
+  envelope field names, raw provider message ids, undecoded `\uXXXX`
+  escapes — occasionally leaked into user-facing prose instead of being
+  summarized. New `gaia_agent_email.answer_grounding` module runs
+  deterministic post-checks on the final answer text at the
+  `process_query` output boundary: an ungrounded success claim or a
+  claim contradicted by the turn's own tool result gets replaced with a
+  grounded fallback, and scaffolding leaks get stripped in place.
+  `check_followups` now also returns an explicit `count` field so nothing
+  is left to miscount. The system prompt's pre-scan coverage note was
+  rewritten to state `scanned`/`total_unread` as two separate facts
+  rather than a "X of Y unread" fraction (matching the attention card's
+  own wording), and to forbid the cross-mailbox phrasing outright. Also
+  root-caused the unicode-escape leak: every `json.dumps` call in the
+  shared agent loop that builds model-visible tool-result text was
+  missing `ensure_ascii=False`, so non-ASCII characters in email subjects
+  reached the model as literal escape sequences — as a side effect, this
+  could also inflate a payload's measured length enough to trigger
+  truncation it did not actually need. Fixed at every call site in
+  `src/gaia/agents/base/agent.py`, benefiting every agent built on it, not
+  just email.
+- **Chat prose no longer contradicts the attention card already on screen
+  (#2636, the other half of the fix above).** The bug as originally filed
+  wasn't the pre-scan guard's territory: the attention card the Go TUI
+  renders (`GET /v1/email/attention`, sections MEETING PROPOSALS/NEEDS
+  REVIEW/ACTION ITEMS) could show real items while the same turn's answer
+  said "no urgent or actionable items found" — because that view is never
+  a tool call (`build_attention_view_impl` has no `@tool` wrapper; it only
+  serves the TUI's on-open render), so the model generating the answer had
+  no way to see it. `answer_grounding` now also reconciles the final
+  answer against the same in-process cache the card was rendered from
+  (extracted into a small new `attention_cache.py` so this stays possible
+  without pulling FastAPI into the dependency-light grounding module), and
+  appends — rather than replaces — a correction naming the card and its
+  coverage when the two disagree. Declines to correct once that cache is
+  older than its own freshness window (120s), so a card the user has since
+  cleared can't get "corrected" back into looking unresolved.
 - **`gaia email autonomy kill` now actually stops a scheduled cycle, not
   just a REST/CLI session's (#2649).** The scheduler builds a brand-new,
   stateless agent from environment variables on every fire and never

hub/agents/email/python/gaia_agent_email/agent.py

@@ -45,6 +45,7 @@ class never passes ``use_claude=True`` / ``use_chatgpt=True`` to
     task_store,
     trust,
 )
+from gaia_agent_email.answer_grounding import ground_final_answer
 from gaia_agent_email.config import ConfigurationError, EmailAgentConfig
 from gaia_agent_email.model_select import (
     NPU_EMAIL_MODEL_ID,
@@ -61,12 +62,7 @@ class never passes ``use_claude=True`` / ``use_chatgpt=True`` to
 )
 from gaia_agent_email.supervision import is_daemon_supervised
 from gaia_agent_email.tools.briefing_tools import BriefingToolsMixin
-from gaia_agent_email.tools.calendar_tools import (
-    CalendarToolsMixin,
-    _listed_event_count_from_conversation,
-    append_conflict_grounding_correction,
-    response_has_ungrounded_conflict_claim,
-)
+from gaia_agent_email.tools.calendar_tools import CalendarToolsMixin
 from gaia_agent_email.tools.connection_tools import ConnectionToolsMixin
 from gaia_agent_email.tools.delete_tools import DeleteToolsMixin
 from gaia_agent_email.tools.followup_tools import FollowupToolsMixin
@@ -221,7 +217,11 @@ def _detect_targeted_mailboxes(query: str) -> set:
   require confirmation.
   check_followups flags sent mail still awaiting a reply; it only reports —
   never draft or send a follow-up nudge unless the user explicitly asks, and
-  any send remains confirmation-gated.
+  any send remains confirmation-gated. Its result's ``count`` field is the
+  exact size of ``awaiting_reply`` — state that number verbatim and list
+  EVERY entry individually; never summarize, merge, or silently drop entries
+  to make a long list feel shorter, and never report a count you arrived at
+  by eyeballing the list yourself.
   list_waiting_on_you flags INBOUND mail awaiting the user's reply (the
   opposite direction from check_followups) — it only reports, and only
   qualifies a message when it has both a genuine ask/meeting-time signal
@@ -279,6 +279,17 @@ def _detect_targeted_mailboxes(query: str) -> set:
   forget the user's writing style from their Sent mail. Local-only:
   reads mail, sends nothing; the profile is stored on-device.
 
+A TOOL CALL IS THE ONLY WAY SOMETHING HAPPENED:
+Never tell the user a mutation (archived, starred, marked read/unread,
+trashed, labeled, moved, quarantined, restored, sent, forwarded, scheduled,
+snoozed, ...) is done, in progress, or confirmed unless you called the
+matching tool THIS turn and its envelope came back ``ok``. If you intend to
+perform an action, call the tool FIRST — its result, not your own
+narration, is what tells the user it happened. A long conversation may
+contain earlier replies where you said "X has been done"; that phrasing
+from a prior turn is never a reason to reuse it for a new request without
+placing a new, matching tool call first.
+
 PRE-SCAN BEHAVIOR:
 When the user asks for a pre-scan, morning brief, triage view, or "what's
 in my inbox", call ``pre_scan_inbox``. The chat surface renders a
@@ -308,7 +319,15 @@ def _detect_targeted_mailboxes(query: str) -> set:
 only covered a fraction. When a mailbox failed (``degraded`` is true /
 ``mailbox_errors`` is non-empty), say so plainly — e.g. "Outlook couldn't
 be scanned (token expired); results below are Gmail only." Never phrase a
-partial scan as if it were a whole-inbox claim.
+partial scan as if it were a whole-inbox claim, and state which of your
+own tools' results you're summarizing (a pre-scan, a briefing, a search)
+so the reader knows what the coverage note refers to.
+
+Never claim "no urgent items" / "no actionable items" / "nothing needs
+you" unless the corresponding list in the result you just received (
+``urgent``, ``actionable``, ``needs_review``) is actually empty — a
+message you are calling out as needing a closer look is not "nothing",
+so name it instead of folding it into an all-clear sentence.
 
 ALWAYS write at least one sentence of plain prose in your final answer. A
 render payload (a ```email_pre_scan fence or any raw JSON) must NEVER stand
@@ -405,8 +424,16 @@ def _detect_targeted_mailboxes(query: str) -> set:
 OUTPUT:
 Tool results come back as JSON envelopes ``{"ok": true, "data": ...}``
 or ``{"ok": false, "error": "..."}``. Summarize tool output briefly for
-the user — do not recite raw JSON. Write plain text only: use Unicode
-symbols directly (→, ≤, ×), never LaTeX/TeX markup like $\\rightarrow$.
+the user in your own words — never recite raw JSON, envelope field names
+(``suggested_archives``, ``needs_review``, ``totals``, ...), or raw
+provider message ids; describe the sender/subject instead, since a
+message id has no reader value. Earlier turns may carry a bracketed note
+about what a card already showed the user, added so YOU can resolve
+"that one" back to a message — that note is for your own reference only,
+never something to quote or repeat verbatim in a new reply. Write plain
+text only: use Unicode symbols directly (→, ≤, ×), never LaTeX/TeX markup
+like $\\rightarrow$, and never leave a backslash-u escape sequence
+unresolved — always write the actual character it represents.
 """
 
 
@@ -460,24 +487,6 @@ def _sub(m: "re.Match[str]") -> str:
     return _LATEX_CMD_RE.sub(_sub, text)
 
 
-def _tool_names_from_conversation(conversation: List[Dict[str, Any]]) -> List[str]:
-    """Every tool name invoked this turn, in call order.
-
-    Mirrors the base ``Agent``'s own max-steps tool tally
-    (``_generate_max_steps_message``) — same shape, read for a different
-    purpose (grounding checks rather than a step-limit summary).
-    """
-    names: List[str] = []
-    for msg in conversation:
-        if msg.get("role") != "assistant":
-            continue
-        for call in msg.get("tool_calls") or []:
-            name = ((call or {}).get("function") or {}).get("name")
-            if name:
-                names.append(name)
-    return names
-
-
 # Redact common credential/token shapes out of a per-row autonomy failure's
 # exception text before it leaves the process (#2625 — adversarial C5).
 # Provider/HTTP client exceptions routinely embed request/response text —
@@ -1019,17 +1028,12 @@ def process_query(self, user_input: str, *args, **kwargs):
         # consumers never see raw TeX in the final answer (#2115).
         if isinstance(result, dict) and isinstance(result.get("result"), str):
             result["result"] = _normalize_plain_text_answer(result["result"])
-            # Flag a conflict verdict the model narrated itself instead of
-            # getting from detect_calendar_conflicts (#2571).
-            conversation = result.get("conversation") or []
-            tool_names = _tool_names_from_conversation(conversation)
-            listed_event_count = _listed_event_count_from_conversation(conversation)
-            if response_has_ungrounded_conflict_claim(
-                result["result"], tool_names, listed_event_count
-            ):
-                result["result"] = append_conflict_grounding_correction(
-                    result["result"]
-                )
+        if isinstance(result, dict):
+            # Single deterministic post-check hook: success-claim / negative-
+            # claim / cross-mailbox / scaffolding-leak / calendar-conflict
+            # (#2571) / attention-card (#2636) guards all live in
+            # answer_grounding.py.
+            result = ground_final_answer(result)
         return result
 
     def _mailbox_target_guard(self, user_input: str) -> Optional[Dict[str, Any]]: