fix(email): "0 messages" for mail that exists — unsupported newer_than: duration unit

[itomek]

Closes #2830

Asking the email agent "how many emails from X in the last two weeks?" would sometimes answer 0 for mail sitting in the inbox, then list thirteen of those same messages a turn later. The cause is that the model non-deterministically writes the time window as newer_than:2w, and w is not a Gmail duration unit — Gmail returns an empty result set for it with no error, so the agent faithfully reports "no messages". Unsupported-but-unambiguous windows are now converted (2w → 14d), genuinely unparseable ones raise an actionable error instead of a silent zero, and the effective query is written to the log so the next empty search takes one line to diagnose instead of a multi-turn investigation.

Important

The root cause in the issue body is wrong, and this PR does not implement what it asks for. #2830 blames the zero-result operator-retry gate and asks for it to be widened. That premise is disproven by measurement, and the widening is deliberately not implemented — see below. This is AC 1's own stated alternative ("a stated, tested reason why an operator query must not be widened"), exercised knowingly.

Evidence

Reproduced live on main, three adjacent turns in one gaia email -i -v session against a real mailbox. Turns 1–2 emitted newer_than:14d and answered 13. Turn 3, the identical question as turn 1, emitted:

🔧 search_messages({"query": "from:\"The Neuron\" newer_than:2w"})
✓ → {"messages": [], "count": 0, "operator_retry": null}   latency_ms: 184.9
→ "there are currently no messages from The Neuron ... over the last two weeks."

The issue reports 191 ms empty vs 2209 ms populated; this run measured 184.9 ms empty. The instability is the model's unit choice, not the mailbox — six single-shot runs all emitted 14d and all passed, which is why this reads as "sometimes it says zero".

After the fix, the same query is normalized and logged (real output):

search_messages effective_query='from:"The Neuron" newer_than:14d' retry=none
search_messages effective_query='from:[REDACTED] newer_than:14d' retry=none
ValueError -> search_messages: cannot parse duration value '1.5w' for the 'newer_than:' operator. Use ...

Why the issue's stated cause is wrong, and the measured accept-list

Measured against the live mailbox and the exact senders the issue cites:

Query	Threads
from:"The Neuron" newer_than:14d	13
from:"The Neuron" newer_than:2w	0
from:"Last Week in AI" newer_than:14d	2
from:"Last Week in AI" newer_than:2w	0

The issue states from:"Last Week in AI" "legitimately matches nothing" because the brand is absent from the sender address. It returns 2 — Gmail's from: matches display names. The premise is false on its own example.

Gmail's real duration grammar, measured (its documentation omits h entirely):

• Accepted, passed through byte-identical: 14d, 14D (case-insensitive), 12h, 336h, 1m, 1y
• Silently returns zero, no error: 2w, 2W, 1.5d, bare 14

Only w is converted — a week is exactly seven days, so it is lossless. Already-valid values are never re-cased; rewriting 14D to 14d would be a no-op that only adds a way to be wrong.

Why the operator-retry widening is NOT implemented

1. Its motivating example is disproven — from:"<brand>" returns 2, not 0 (above).
2. It would not have fixed this bug. The reproduction is newer_than:2w; a segment-preserving widening keeps that term and still returns zero, and a whole-query wrap produces from:(from:"X" newer_than:2w) OR …, which also returns zero.
3. It would not have fixed the one other real defect observed — a run emitted subject:'Last Week in AI' and answered 1 instead of 2. Wrong, but non-zero, so a zero-result retry never fires.
4. reply_tools.py:297-298 carries an independent copy of the same gate on the side-effecting draft_reply target path, with no opt-out flag. A bad widen there attaches a draft to the wrong thread.

operatorize_query and has_gmail_operator therefore keep their exact behaviour, test_operator_query_never_retried stays green, and a new test pins that the reply path still makes exactly one backend call for a zero-hit operator query. Follow-up issue to be filed for a targeted widening covering both call sites.

Also in this PR

• _redact never matched email addresses. ~/.gaia/gaia.log is bundled by gaia diagnostics by default and the docs tell users to attach it to a GitHub issue, so logging queries without fixing this first would put contacts' addresses on a path to public disclosure. Fixed at the shared primitive, mirroring agent.py's existing _AUTONOMY_ERROR_EMAIL_RE (fix(email-agent): one per-message error aborts the whole autonomy cycle and discards the report #2625/C5). This is a prerequisite for the logging, not a drive-by.
• POST /v1/email/search returned a bare 500 for an unparseable operator value — the route caught only the four ConnectorsError subtypes. Now an actionable 400.
• The test fixture accepted w as a valid unit, modelling the bug as correct behaviour — plausibly why this shipped at all. It no longer accepts what Gmail rejects.

Outlook is unaffected. outlook_backend.py passes the whole query as a quoted Graph $search phrase and parses no Gmail operator, so the corrected duration is inert there. Outlook's operator search was already non-functional; out of scope here.

Test plan

• python -m pytest hub/agents/email/python/tests/ tests/unit/agents/email/ -q → 2657 passed (needs pip install build for the sdist test; otherwise 1 unrelated env failure)
• Live reproduction on main before the fix, and normalization + redaction + error path exercised after (output above), on macOS/arm64 with Gemma-4-E4B-it-GGUF on Lemonade
• Real-world re-run of the multi-turn session to confirm the false zero no longer occurs

[itomek]

Real-world evidence — macOS/arm64, live Gmail, Gemma-4-E4B-it-GGUF on Lemonade

The fix is proven against LiveGmailBackend (the real Gmail API), not a fixture.

The failing query now resolves correctly. Ground truth for this mailbox today is 4 messages in the last two weeks; raw Gmail returns 0 for the 2w form:

Input to search_messages	Effective query (new log line)	Result
from:"The Neuron" newer_than:14d	newer_than:14d	4
from:"The Neuron" newer_than:2w	newer_than:14d	4
(same 2w query straight to the Gmail API, unfixed)	—	0

search_messages effective_query='from:"The Neuron" newer_than:14d' retry=none
  INPUT 'from:"The Neuron" newer_than:14d'         -> count=4
search_messages effective_query='from:"The Neuron" newer_than:2w'  -> normalized
  INPUT 'from:"The Neuron" newer_than:2w'          -> count=4

Full agent turn, live, in-process (latency=679ms — a real Gmail round-trip):

tool_call name=search_messages
search_messages effective_query='from:"The Neuron" newer_than:14d' retry=none
tool_result name=search_messages ok=True latency=679ms
→ "There are 4 messages from The Neuron in your inbox over the last two weeks."

Verified against the mailbox independently: from:"The Neuron" newer_than:14d returns exactly 4 threads. The answer is correct.

Redaction and the loud-error path, same live code path:

search_messages effective_query='from:[REDACTED] newer_than:14d' retry=none
ValueError -> search_messages: cannot parse duration value '1.5w' for the
              'newer_than:' operator. Use an integer plus h/d/m/y ...

Note the address is redacted while newer_than:14d survives — the diagnostic signal is preserved, which is the whole point of the log line.

Tiers

• Unit / integration — 2657 passed, 0 failed (hub/agents/email/python/tests/ tests/unit/agents/email/). One sdist test needs pip install build; without it that single test errors on a missing module, unrelated to this change.
• Real-world — above, on macOS/arm64 with live Gmail and a live local model.
• Not covered: the daemon-relay path (gaia email -q) could not be exercised — the daemon-spawned agent subprocess is denied macOS Keychain access (-128) in a non-interactive session, though the same code works in-process. That is an environment limitation of this host, not a code path this PR changes.

One honest caveat

The trigger is stochastic — the model emits 2w only some of the time, so this run's agent turn happened to emit 14d. Rather than wait on model luck, the table above feeds the 2w string directly through the fixed code to the live API, which is the deterministic form of the same proof: the input that used to return 0 now returns 4.

[github-actions]

Verdict: Approve with suggestions

A "last two weeks" mail search used to come back empty even when the mail was there — Gmail doesn't understand weeks and says nothing about it. This PR converts weeks to days before the query is sent, rejects genuinely malformed durations with a message that tells you what to type instead, and makes the test mailbox stop accepting a unit real Gmail rejects. That last part is the good bit: the fake was lying, which is why the bug survived a green test suite.

Nothing here blocks merge. Two things are worth a look before it lands:

• The new error handler on the search endpoint is wider than the problem it fixes. It wraps the whole search operation, so an internal failure anywhere inside it — a malformed message coming back from the mailbox, for instance — will now be reported to the caller as "your request was bad" with the raw internal message attached, rather than as a server error. Narrow it to just the query-parsing step.
• A duration wrapped in brackets now fails hard where it used to work. If a query ever groups the recency filter in parentheses or braces, the closing bracket gets swallowed into the value, and a search Gmail would happily run turns into an error instead. Uncommon shape, easy guard.

Real-world evidence

Strong, and it carries the verdict. The bundle shows the real sidecar run over HTTP on both this branch and the merge-base, same request, same mailbox: newer_than:2w went from 0 hits to the correct single hit, older_than:2w from empty to the right 40-day-old message, and newer_than:1.5d from a silent empty 200 to a 400 that names the bad value and the accepted units. Two planted messages at different ages confirm the converted query is still a real time filter rather than a widened match. The CLI entry, the sibling routes, and the after: date family were spot-checked alongside it — the last of which turns out to be repaired by the same handler.

Deferred, with the reason stated: the new log line and its redaction, and any Agent UI pixels, both need real inference and are marked pending the strix-halo lane. Those two surfaces rest on static review here. The verified surface is the one the bug actually lived on, so I'm treating the evidence as adequate for merge.

🔍 Technical details

🟡 Important

1. except ValueError on the search route is too broad (api_routes.py:2231)

The handler wraps the entire asyncio.to_thread(_search_inbox, …) call, not just the normalizer. _search_inbox also does backend.list_messages, backend.get_message, _format_message_for_llm, and — critically — constructs EmailSearchResultItem / EmailSearchResponse. Pydantic's ValidationError subclasses ValueError, so a message whose From/To/date header parses to None against a str-typed field now returns 400 with a pydantic error dump in detail, telling the caller their query was malformed when it wasn't, and leaking internal field structure.

Normalize before the try so only the intended ValueError is caught:

    try:
        normalized_query = (
            normalize_gmail_date_operators(request.query) if request.query else request.query
        )
    except ValueError as e:
        # An unparseable date/duration operator value -- an actionable 400,
        # never a bare 500 (#2830).
        raise HTTPException(status_code=400, detail=str(e)) from e
    try:
        return await asyncio.to_thread(
            _search_inbox,
            backend,
            query=normalized_query,
            labels=request.labels,
            max_results=request.max_results,
            page_token=request.page_token,
        )

(That needs normalize_gmail_date_operators imported at the route and _search_inbox to skip re-normalizing — or, if you prefer to keep the lazy import inside _search_inbox, split it into a _normalize_query helper called from the route.)

2. A bracketed duration operand now raises where it previously worked (gmail_query.py:36)

_DURATION_OP_RE's \S+ fallback is greedy, so ... OR (newer_than:7d) yields the value 7d), which fails _DURATION_VALUE_RE.fullmatch and raises. Gmail accepts () and {} grouping, so a query that works today becomes a hard error. _DATE_OP_RE has the same shape of gap for after:2026/01/01), so this isn't new in kind — but it is newly reachable for the duration family, and it fails loud rather than passing through.

Cheapest guard is to strip the grouping characters in _parse_gmail_duration_value alongside the quotes:

    value = raw.strip().strip('"').strip("(){}").strip()

Worth a parametrized case ("(newer_than:7d)", "{newer_than:7d}") next to the existing space-after-colon gap test.

🟢 Minor

3. The effective-query log is skipped on the path you most want it for (read_tools.py:977) — it sits inside the with log_tool_call(...) block, after both list_messages calls. If the backend raises mid-search, no greppable query line reaches ~/.gaia/gaia.log, which is exactly the "user reports a bad search, reproduce it from the bundle" case the function was added for. Moving the call to just before the first list_messages (retry state can follow in the existing result_summary) or into a finally would close it.

4. The CHANGELOG doesn't mention the new email-address redaction. _REDACT_PATTERNS is package-wide — this changes tool_call / tool_result output for every tool, not just search_messages. send_email's recipient in a verbose log is now [REDACTED], which is a deliberate privacy win but also a debuggability change anyone reading old logs will notice. Per CLAUDE.md's doc rule it belongs in the entry.

5. Underscore-prefixed names imported across modules (read_tools.py:45) — _DURATION_OP_RE and _parse_gmail_duration_value are private by name but public by use; gmail_query.py exists specifically to be imported from read_tools. Dropping the underscore on both (or exporting a single normalize_duration_operators(query) and keeping the regex genuinely private) matches the intent.

6. Hardcoded line count in a docstring (gmail_query.py:16) — "already 3021 lines" is true today and wrong after the next edit to read_tools.py. The rationale stands without the number:

A peer module rather than living in ``tools/read_tools.py`` (already very
large) -- imported from there, which is the sole call site that wires it into

7. Worth confirming: does a new 400 trigger on POST /v1/email/search warrant a SCHEMA_VERSION bump? CONTRACT.md states the minor bumps "on every change, additive or breaking." Every existing entry in the version history is a field/value change and 400 is already declared on the route via _AMBIGUOUS_PROVIDER_400, so 2.14 may well be correct — but the new condition is observable to a consumer, and a deliberate "no bump" is worth saying out loud.

Strengths

• Fixing the fixture is the actual root-cause move. Removing w from _RELATIVE_UNIT_SECONDS in fake_gmail.py is why this bug can't come back the same way — the fake was accepting a unit the real API silently rejects, which is precisely the "mocks prove we called it, not that the call is valid" trap CLAUDE.md calls out.
• test_fixture_query_matches_rejects_week_unit_explicitly pins the fixture's own behaviour, with a comment explaining that _query_matches reaches its free-text branch by fall-through rather than by an explicit rule. That's the test most people wouldn't write, and it's the one that guards the guard.
• The known gaps are pinned rather than papered over — test_duration_space_after_colon_is_a_known_unvalidated_gap documents the newer_than: 2w shape as accepted-and-unvalidated with a reason for not widening the regex, and the CHANGELOG states plainly that Outlook is unaffected and why. Both are the honest version.
• The w→days conversion is lossless and case-preserving, and _DURATION_OP_RE is deliberately scoped to the exact _than operator names so it can't misparse a newer:/older: date as a duration — with the comment explaining why the broader alternation was rejected.

[itomek]

All seven addressed — two code fixes, four doc/naming, one deliberate no-change. Pushed in 4131004b and 0083de6c.

On #2, I did not take the suggested fix. .strip("(){}") stops the raise but corrupts the query: the substitution rebuilds the token as newer_than:7d, so the stripped bracket is never re-emitted and (newer_than:7d) becomes (newer_than:7d — unbalanced. Excluding the grouping characters from the capture instead means the bracket never enters the match, so it survives untouched:

'(newer_than:7d)'            -> '(newer_than:7d)'      (was: ValueError)
'{from:a newer_than:2w}'     -> '{from:a newer_than:14d}'
'from:x OR (newer_than:2w)'  -> 'from:x OR (newer_than:14d)'

Good catch — this was a genuine regression I introduced, and it would have hard-failed queries that work on main today.

Per-finding detail

#	Action
1	Fixed. Query normalization split into _normalize_search_query and called from the route; only that is wrapped in except ValueError → 400. Two tests pin it: an unparseable duration is a 400, and a ValueError raised inside the backend is not reported as a bad request.
2	Fixed, differently — see above. Parametrized cases added for (), {}, ], and grouped 2w conversion.
3	Fixed. The log call moved into a finally, so a backend raise still leaves a greppable query line.
4	Fixed. CHANGELOG now has a ### Changed entry stating the redaction is package-wide, that a send_email recipient in a verbose log is now [REDACTED], and that pre-existing logs still hold raw values.
5	Fixed. DURATION_OP_RE / parse_gmail_duration_value — public by use, now public by name.
6	Fixed. Line count dropped from the docstring.
7	Deliberate no-bump, stated as requested — reasoning below.

Why no SCHEMA_VERSION bump. No field, shape, or response code changed: 400 was already declared on /search. What changed is when a 400 occurs, inside an already-declared code — and every existing row in the version history is a field or value change. I tried the bump first; it cascaded into 8 pinned tests plus regenerating two committed artifacts, one of which has a known trap where regeneration deletes a section (#2093). That is real risk on a bugfix PR in exchange for a version number that signals nothing new to a consumer.

What was wrong is that the route's 400 description only documented "ambiguous or unknown account", so the new condition was undeclared. That description now names it, and openapi.email.json is regenerated — a one-line diff, no section loss. Happy to bump if you read the every-change rule more strictly than I have.

Not addressed, deliberately: read_tools.py has pre-existing black drift in check_suspicious_mail (~line 3157) that util/lint.py misses because it does not cover hub/. It is unrelated to this change and reformatting it here would be a drive-by. Worth its own PR, along with extending the lint entry point to hub/.

Suite: 2666 passed, lint clean.

[github-actions]

Skill audit

Skill	Verdict	Claimed tier	Cleared tiers	Findings	Rules
.claude/skills/testing-the-gaia-agent	✅ ALLOW	experimental	experimental, community	none	—
hub/agents/gaia/npm	✅ ALLOW	experimental	experimental, community	none	—
hub/agents/gaia/python/gaia_agent/skills/gaia-voice	✅ ALLOW	community	experimental, community	none	—
hub/skills/check-in	✅ ALLOW	community	experimental, community	none	—
hub/skills/coding	✅ ALLOW	community	experimental, community	1 info	permission.unused
hub/skills/daily-brief	✅ ALLOW	community	experimental, community	1 info	permission.unused
hub/skills/data-explore	✅ ALLOW	community	experimental, community	none	—
hub/skills/document-brief	✅ ALLOW	community	experimental, community	none	—
hub/skills/github-triage	✅ ALLOW	community	experimental, community	1 info	permission.unused
hub/skills/price-watch	✅ ALLOW	community	experimental, community	1 info	permission.unused
hub/skills/recommendations	✅ ALLOW	community	experimental, community	1 info	permission.unused
hub/skills/research-report	✅ ALLOW	community	experimental, community	1 info	permission.unused
hub/skills/rss-digest	✅ ALLOW	community	experimental, community	2 info	code.suppression, permission.unused
hub/skills/source-watch	✅ ALLOW	community	experimental, community	1 info	permission.unused

✅ All audited skills cleared the tier they claim.

Per-finding detail is withheld here on purpose. Read it in the Security > Code scanning tab, or download the skill-audit-reports artifact from this run. Offending source text is withheld from CI everywhere — reproduce it locally with gaia skill audit <dir> --show-snippets.