fix(email-agent): search_messages returns no answer at all for a long-bodied sender — context overflow

[itomek]

Problem

Ask the email agent about a sender whose messages have long bodies and it returns no answer at all — not a wrong answer, nothing. The search succeeds, the results come back, and the model then blows the context window and emits a canned apology.

Verbatim, every time:

I had to trim the conversation to fit my memory but I'm still not making progress. Could you re-ask in a fresh chat with just the essentials?

Reproduced independently on two machines and two operating systems, so this is not platform-specific. Both GPU-only and Gmail-only, both Gemma-4-E4B-it-GGUF at ctx 65536 (runtime-resolved, verified — not the configured value):

box	probe	true	runs	result
macOS / Metal	from:Every newer_than:14d, both phrasings	15	2/2	tool succeeded, then the fallback. No answer.
Linux / vulkan	from:Every, 3 REST + 3 TUI	15	6/6	stated 0, enumerated 0. No answer.

The tool call itself is fine in every run — tool_end success=true, ~1.8s, results returned. Determinism is total: 8 of 8 across both boxes, zero variance.

Root cause path (src/gaia/agents/base/agent.py:3758 streaming, :3907 non-streaming): the LLM call raises, is_ctx_overflow is true, the loop calls _shrink_messages_for_overflow(messages) and retries exactly once (_retried_after_trim_stream), overflows again, and emits that string as final_answer. The fixed single retry is what makes the failure perfectly deterministic — a payload-size cliff alone would show variance near the boundary.

The mechanism underneath is NOT yet measured, and that is the first task here. search_messages sizes its payload via _format_messages_within_budget against envelope_budget_tokens(ctx_size=active_profile_ctx_size()) — 65536 minus the fixed prompt and response reserve — and can shrink bodies to a 200-char floor per message (THREAD_MIN_PER_MESSAGE_CHARS, hub/agents/email/python/gaia_agent_email/tools/read_tools.py:93). Fifteen messages at that floor is ~3K chars, which fits trivially. So either the shrinker is not firing on this path, or the envelope budget ignores what the rest of the conversation already occupies. Do not assume which — measure it.

Two contributing factors worth carrying into the fix:

1. search_messages ships full message bodies — DEFAULT_BODY_LIMIT_CHARS = 4000 per message (read_tools.py:79) — for questions like "how many emails from X" that need zero body bytes. The bodies are the entire payload cost.
2. The model widens its own request. In the macOS runs it chose max_results of 100 and 50, not the 25 default, because the phrasing implied breadth ("all messages", "how many"). It asks for more than it can fit.

Part of #2511 (surfaced through T5). Distinct from #2756 / #2760, which fix the wrong count case — a precomputed count cannot help a turn that never reaches the answering stage. #2760 states this boundary explicitly rather than claiming T5 fixed.

Outcome

A counting or listing question against a long-bodied sender returns a real answer instead of the context-overflow apology, on the same GPU/Gmail conditions where it currently fails 8 of 8.

Acceptance criteria

• Root cause measured and stated before any fix. For the failing probe, record the serialized envelope size in chars and estimated tokens, whether _format_messages_within_budget fired, the per-message body limit it settled on, and the budget it computed. Expected artifact: a comment on this issue naming which of the two hypotheses holds — shrinker did not fire or budget ignores existing conversation occupancy — with the numbers behind it.
• The failing probe answers. Given from:Every newer_than:14d (true 15, Gmail-only, GPU) → the agent states 15, on 3 consecutive runs, with no occurrence of the string "I had to trim the conversation". Today: 0 of 8 runs answered.
• A metadata-only path exists for counting questions — a counting/listing ask must not pull 4000-char bodies. Expected: for the same probe, the tool payload contains no message body content, and the envelope size drops by at least an order of magnitude versus today's measured value from criterion 1.
• The no-answer path is never silent about why. If a turn still cannot fit, the user-facing text names the actual constraint and a next step, not a generic "re-ask with just the essentials". Expected: the message states the result set was too large and suggests narrowing the query (e.g. a shorter date range).
• Unit tests cover the payload-size guard at the registered-tool layer: a seeded backend returning N long-bodied messages produces an envelope under the computed budget. Assert the envelope size, not just that the call returned.
• Live-inference check on GPU + Gmail, since the defect is model-context behaviour that unit tests cannot reach. Record the runtime-resolved model id and device, the model's chosen max_results, and the verbatim answer for each of the 3 runs.

Scope & expectations

• Where: hub/agents/email/python/gaia_agent_email/tools/read_tools.py (search_messages wrapper :2225-2317, search_messages_impl :841, _format_messages_within_budget :176), hub/agents/email/python/gaia_agent_email/context_budget.py, and the overflow path in src/gaia/agents/base/agent.py:3758 / :3907.
• Out of scope: the wrong-count defect (fix(email-agent): search_messages states a wrong, unstable count for an intact result set #2756, fixed in fix(email-agent): search_messages states an exact, stable message count #2760); multi-mailbox aggregation semantics; check_followups, which ships compact metadata rows and does not exhibit this.
• Constraints: GPU-only and Gmail-only for evidence; no new dependencies; this repo's fail-loudly rule applies — a silently truncated result set presented as complete is not an acceptable fix.

How to verify

CLI/agent surface, 3 runs of the currently-failing probe:

gaia tui chat --agent email --query 'how many emails from Every in the last two weeks?'

Attach per run: the verbatim answer, the tool_args the model chose (including max_results), the measured envelope size, and the runtime-resolved model id + device. Plus the ground-truth count from POST /v1/email/search for the same query string.

[itomek]

Root cause: H2 — the envelope budget ignores what the rest of the conversation already occupies. _format_messages_within_budget does fire on every search_messages call — that part works. The budget it's handed to shrink against is inflated by roughly 40K tokens, so its shrink branch never triggers, and the unshrunk envelope goes out the door still far too big for the real window.

Measured numbers (hermetic repro: real EmailTriageAgent, full tool registry, FakeGmailBackend seeded with 15 messages from one sender with 12,000-char raw bodies, GPU profile ctx=65536 — same numbers the code itself computes, no live Lemonade needed to see the shape of the bug)

The search_messages_impl call itself (max_results 25, 50, and 100 — identical result at every value, since the true count of 15 is already below all three):

_format_messages_within_budget fired?	Yes — it's called on every search_messages_impl invocation
Did its binary-search shrink branch execute?	No — the unshrunk check passed on the first attempt
Per-message body limit it settled on	4000 chars (DEFAULT_BODY_LIMIT_CHARS, i.e. unshrunk)
Serialized envelope	68,784 chars / ~52,911 estimated tokens (estimate_tokens_json)
Budget it computed	envelope_budget_tokens(ctx_size=65536) = 55,296 tokens

52,911 ≤ 55,296, so the code believes this fits comfortably. It doesn't — because 55,296 isn't the real available room.

Why the budget is wrong: envelope_budget_tokens subtracts a hardcoded _AGENT_LOOP_FIXED_TOKENS = 9216 (its own comment: "the system prompt PLUS the full JSON tool schema") plus a 1024-token response reserve from ctx_size. Measuring the REAL fixed cost of a live EmailTriageAgent turn today, using the same estimator the code uses:

Component	Measured
system_prompt (mixins + agent prompt + prose tool summaries)	5,897 est. tokens
_openai_tools — the separate tools= JSON schema Lemonade receives on every turn (65 registered tools, full per-tool docstrings — EmailTriageAgent has no dynamic tool-selection, so it's always the full registry)	43,307 est. tokens
Real fixed overhead	~49,204 est. tokens
Fixed overhead the budget assumes	10,240 (9216 + 1024)

The assumed constant under-counts today's real fixed cost by ~4.8×. _AGENT_LOOP_FIXED_TOKENS's own comment says it was measured once against a #2087 CI run on a different (bulk-triage) path — it has never been re-measured as the tool registry grew to 65 tools, and it has no live connection to the actual _openai_tools payload, so every tool added since silently erodes its accuracy further.

Real headroom for a tool-result envelope is 65536 − 49204 − 1024 = 15,308 tokens, not 55,296 — a 3.6× gap. The unshrunk 52,911-token envelope is ~3.5× bigger than that real headroom.

Stacking the actual probe's full request (system prompt + the full 65-tool schema + the user's question + the model's tool call + the search_messages result): ~102,262 estimated tokens against the 65,536 ctx window — a ~36,726-token (56%) overflow, on the first and only tool call. No multi-step accumulation is needed to blow the window; one search_messages on a long-bodied sender does it alone.

H1 is false in the strict sense ("not firing on this path at all") — _format_messages_within_budget runs on every call. The real defect is one level up: the budget it's told to shrink against doesn't reflect what the turn actually costs, because that cost is a stale hardcoded constant instead of a measurement of the live system prompt + tool schema. Fix direction: the budget calculation needs to account for what's actually already committed for this turn, not a flat number picked once and never revisited — plus the metadata-only counting path and the fail-loud message named in the acceptance criteria.

[itomek]

Correction to the numbers above. The estimates in my previous comment used this repo's own char-based estimate_tokens_json (deliberately pessimistic — its own docstring floors at 1.3 chars/token). Cross-checking the exact same payload strings against Gemma-4-E4B-it-GGUF's real tokenizer (llama-tokenize against the actual local GGUF, the same tokenizer Lemonade uses under the hood) changes the magnitude of the finding — reporting the corrected numbers rather than let the estimate stand uncorrected in this issue.

Real-tokenizer numbers (same 5 payload strings, llama-tokenize -m gemma-4-E4B-it-Q4_K_M.gguf --show-count)

Component	Repo estimator (estimate_tokens/_json)	Real Gemma tokenizer
system_prompt	5,897	5,996
_openai_tools (65-tool schema)	43,307	14,826
user query	13	12
assistant tool_call	91	43
search_messages result (15 long-body messages)	52,954	13,804
Total, one call	102,262	34,681

The repo's pessimistic estimator overshoots the real tokenizer by ~2.9× on the tool schema and ~3.8× on the message envelope — both are JSON-structural content (lots of braces/brackets/short keys), which compresses far better under real BPE than the estimator's "densest plausible ASCII JSON" floor assumes. On prose (system_prompt, the user query) the estimator is accurate to within 2%.

Corrected conclusion: real fixed overhead (system_prompt + the 65-tool schema) is 20,822 tokens — 2.0× the _AGENT_LOOP_FIXED_TOKENS + response reserve constant (10,240), not the ~4.8× I stated above. Real headroom for a tool-result envelope is 65536 − 20822 − 1024 = 43,690 tokens, vs. the 55,296 the code computes — the code overestimates room by ~11.6K tokens, not ~40K.

With that correction, a single search_messages call on 15 long-body messages (13,804 real tokens) does NOT overflow the real 65,536 window — total ~34,681 tokens, 47% headroom. My original claim that one call alone overflows by 56% does not survive the real-tokenizer check and I'm retracting it. _AGENT_LOOP_FIXED_TOKENS being under-measured is still real and still directionally H2, just smaller than first stated.

Repeating the same real numbers for N search_messages calls accumulating in one turn (nothing prunes an earlier tool result until the one-shot overflow-retry fires, agent.py:2649):

calls in one turn	real tokens	fits 65,536?
1	34,681	yes (47% headroom)
2	48,528	yes (26% headroom)
3	62,375	yes (barely — 4.8% headroom)
4	76,222	no — overflows by ~10.7K

Given a clean single call doesn't reproduce the overflow against the real tokenizer using synthetic prose bodies, the observed 8/8 real-hardware failure most plausibly needs one of: (a) real Every.to bodies being denser in real tokens than clean prose once truncated to 4000 chars (HTML/CSS remnants, tracking links, markup that survives normalization), (b) 3+ large tool calls accumulating in the same turn before the model attempts an answer, or (c) the probe landing several turns into an existing chat rather than a fresh conversation, so prior history is also in messages. I can't distinguish between these without the live GPU run, which is a separate, already-required acceptance criterion — I'll report back with the real Lemonade-reported prompt size once that run happens.

None of this changes the fix direction: _format_messages_within_budget's shrink branch is still unreachable-when-it-shouldn't-be because the budget it's handed doesn't reflect what the turn actually costs (mechanism unchanged by this correction — only the size of the resulting gap). The metadata-only counting path remains the right fix for the stated probe regardless of exactly which combination of the above closes the gap, since it removes the dominant cost (bodies) outright rather than needing the budget arithmetic to be exactly right.

[itomek]

Status: root cause not yet fully established — measured enough to rule things out, not yet enough to close AC #1. Recording exactly where this stands before touching the live box, per this issue's own hard rule.

What's ruled out: a single, isolated search_messages call does not overflow. Real-tokenizer total ~34,681 of 65,536 (47% headroom) — my earlier "one call alone overflows by 56%" claim (previous comment) doesn't hold, already retracted.

A structural finding worth stating precisely, even though it isn't the cause: the code's two inputs to its own shrink decision are wrong in opposite directions and nearly cancel for this exact probe.

	wrong (today)	corrected
budget (envelope_budget_tokens)	55,296 (assumes 10,240 fixed)	43,690 (real fixed cost 20,822 + reserve)
envelope (15 long-body messages)	52,954 (repo's char-based estimator)	13,804 (real tokenizer)
shrink fires?	No (52,954 ≤ 55,296, margin 2,342)	still No (13,804 ≤ 43,690, margin 29,886)

Both the wrong accounting and the corrected accounting agree "don't shrink" for a single call — so this coincidence, while a real fragility (a slightly larger single call could flip either input and the two errors would no longer cancel), is not itself what produces the observed 8/8 overflow. Recording it because it's true and diagnostic of how fragile the current arithmetic is, not because it's the answer.

What's now the leading candidate, with independent cross-validation: prior conversation-turn history. #2766, working the same model/box/ctx (Gemma-4-E4B, gpu, 65536) on a sibling issue, found live on this exact stack: a fresh single-turn question answers cleanly; the same question asked as a second turn in an ongoing session fabricates, and picks a different tool entirely. The TUI's context is assembled from prior turns ending in final — so turn history is the variable, confirmed, not hypothesized.

This composes with my own candidate #1 (multiple large tool calls stacking with nothing pruning an earlier one) — a multi-turn session is exactly how that stacking happens without anything pruning it: each prior turn's own tool result rides along as part of context into the next turn's prompt. They may be one mechanism seen from two angles.

Testable prediction, landed before the live run so the run can confirm or refute it rather than just illustrate it: using the corrected real numbers (fixed overhead 20,834 including the user query; each additional search_messages-sized call adds ~13,847 tokens — 43 for the tool call + 13,804 for a 15-long-body-message result):

tool-call-sized additions	total	fits 65,536?
1	34,681	yes (47% headroom)
2	48,528	yes (26% headroom)
3	62,375	yes — barely (3,161 headroom, 4.8%)
4	76,222	no — over by 10,686

Prediction: it takes on the order of 3–4 accumulated large-envelope tool results (whether from multiple calls in one turn, or one call each across multiple turns, or a mix) to cross the real ceiling — not one. If a 2-turn session (one prior tool call + this issue's search_messages call) is enough to reproduce the failure on real hardware, then either the prior turn's own tool result is larger than my ~13.8K-token estimate, the prior turn itself made more than one tool call, or there's session-level context (memory retrieval, style guidance) I haven't sized. The live run settles which.

Diagnostic plan for the live run (not confirmatory — diagnostic, per direction): structure the probe as a realistic multi-turn session, not an isolated first message (a fresh single-turn ask is now known to answer cleanly and would produce a false "can't reproduce"). Capture, at the moment of the failing turn: the sidecar's own log (~/.gaia/host/daemon.log, or gaia daemon logs) for Lemonade's raw overflow error text — which, per the OpenAI-compatible error shape, typically states the real computed prompt_tokens directly, the strongest single number available — plus the structured tool_call/tool_result log lines gaia_agent_email's verbose logging already emits for every tool invocation in the session, to reconstruct exactly how many tool results were resident and what each contributed.

Sampling on this model is temp 1.0 / top_p 0.95 — non-deterministic in general, but the overflow itself is a hard prompt-size check that happens before any sampling occurs, so an overflow is deterministic given the same accumulated context; the 3-consecutive-runs requirement in this issue's AC is about the quality of a successful answer, not about whether overflow recurs.

[itomek]

Filed the estimator-defect finding as its own issue: #2772 (context_budget's token estimator overshoots real tokenization by 2.9-3.8x on JSON payloads — the tool-schema size and the cleared _handle_large_tool_result check are folded in there as context/secondary findings, not separate issues).

[itomek]

Root cause confirmed on real hardware — GPU/Gmail, Gemma-4-E4B-it-GGUF, ctx 65536, live mailbox. Candidate #3 (prior conversation-turn history) is the primary, now-measured mechanism, not a hypothesis.

The reproduction and the ground-truth numbers (real Lemonade error, not an estimate)

Structured as a realistic multi-turn TUI session, since a fresh isolated first turn is now known (via a sibling issue's cross-validation) to answer cleanly and would falsely read as "can't reproduce":

1. Agent startup auto-scan → pre_scan_inbox (→ triage_inbox, detect_waiting_on_you)
2. Turn 1, "What's new in my inbox today?" → pre_scan_inbox again (→ same sub-calls) — answers correctly, UI stats: 44.6s · 2 steps · 1 tools
3. Turn 2, this issue's exact probe verbatim, "how many emails from Every in the last two weeks?" → search_messages (ONE call, not stacked within this turn) — fails, verbatim: "I had to trim the conversation to fit my memory but I'm still not making progress. Could you re-ask in a fresh chat with just the essentials?"

The sidecar log (gaia daemon logs, plus the per-sidecar log under ~/.gaia/agents/email/logs/) carries Lemonade's own error, which states the real computed prompt size directly — the strongest number available, not an estimate:

request (72438 tokens) exceeds the available context size (65536 tokens)
n_ctx: 65536, n_prompt_tokens: 72438

72,438 real tokens against a 65,536 window — over by 6,902 tokens (10.5%). The one-shot shrink-and-retry (agent.py:2649) barely helped: the retried request still failed at n_prompt_tokens: 72357 — a 81-token reduction. This matches reading _shrink_messages_for_overflow's own logic: it keeps the latest tool result (here, search_messages's own large envelope) fully intact and only stubs older ones, so when the newest call is itself the big one, the retry changes almost nothing.

What this settles among the three candidates named in my prior comment:

• Candidate Update Driver Check #3 (prior turn history) — confirmed, primary. Turn 2's own search_messages call made exactly one tool call — not stacked within itself — yet the request still overflowed by 10.5%. The only source for the other ~58K tokens is what carried over from the auto-scan and turn 1's pre_scan_inbox results still occupying the prompt when turn 2 executes. Whatever mechanism assembles cross-turn context is carrying substantially more than a compact summary of prior turns — a single search_messages call alone (measured earlier, real tokenizer) costs ~13.8K tokens, nowhere near enough by itself to explain 72,438.
• Candidate Update installer and workflows/actions for CI/CD #1 (multiple tool calls stacking within one turn) — same effect, different route: it's not literally realized as multiple calls inside one process_query turn here (this turn made one call), but the general mechanism — nothing prunes an earlier large tool result before it counts against the next request's budget — is exactly what's firing, just across separate /v1/email/query requests rather than within a single one. One mechanism, as predicted, seen from two angles.
• Candidate Use public Lemonade hybrid installer #2 (denser real HTML bodies than my synthetic prose) — not needed to explain this failure; real Gmail bodies plus cross-turn accumulation were already sufficient without invoking body density as a separate factor.

What's out of my scope but worth flagging: the actual code path that assembles a later turn's prompt from earlier turns' results (in the TUI/API request layer, not read_tools.py/context_budget.py/the agent-loop overflow path I own) is not something I've traced or am fixing here — it is a different subsystem than search_messages's own envelope budgeting. This issue's fix (the metadata-only search path) directly addresses the largest single lever available at my layer: making the new turn's own tool result negligible in size regardless of how much prior-turn content is already resident, since a metadata-only envelope (~13x smaller, per the earlier order-of-magnitude measurement) leaves far more headroom to absorb whatever carries over from earlier turns.

AC #1 is met: root cause measured and stated, with the numbers, before the fix (already committed on issue-2763, PR to follow) is presented as evidence.