fix(email): don't cancel a retrying agent for a recoverable tool error - #2572
Merged
Conversation
The batch-tool retry path (agent.py's STATE_ERROR_RECOVERY) was being killed by the SSE layer: a per-tool error the loop had already started retrying was indistinguishable from a fatal top-level failure once it reached the wire, so the email sidecar's translator mapped it to a terminal error, the query route ended the stream, and its cleanup then cancelled the still-retrying agent mid-recovery. print_error() now takes a recoverable flag so the SSE event carries it through; the email sidecar's canonical translator folds a recoverable agent_error to a non-terminal status line (same pattern already used for tool_confirm_denied) instead of a terminal error, so the run stays alive and the retry can reach completion. The user still sees the failure — it's surfaced as a status line, not silently swallowed.
Black wanted print_error's line wrapped in src/gaia/api/sse_handler.py.
58 tasks
itomek
marked this pull request as ready for review
July 28, 2026 12:26
itomek-amd
previously approved these changes
Jul 28, 2026
itomek-amd
left a comment
Collaborator
There was a problem hiding this comment.
Verified. Clean threading of a recoverable flag from print_error through every SSE handler to the translator, where a recoverable agent_error folds to a non-terminal status line instead of a terminal error — so a per-tool retry (STATE_ERROR_RECOVERY) no longer gets the still-retrying agent cancelled out from under it. The flag is only set when True, keeping the wire shape unchanged for every existing fatal caller; recoverable: false and absence both stay terminal. The end-to-end route test proving both tool_calls stream and the run reaches final is the strong evidence here.
itomek-amd
previously approved these changes
Jul 28, 2026
itomek-amd
previously approved these changes
Jul 28, 2026
itomek-amd
previously approved these changes
Jul 28, 2026
itomek-amd
approved these changes
Jul 28, 2026
itomek-amd
previously approved these changes
Jul 28, 2026
itomek-amd
previously approved these changes
Jul 28, 2026
itomek-amd
previously approved these changes
Jul 28, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
When the model calls a batch tool with a bad argument (e.g.
archive_message_batchwith a straymailboxkwarg), the agent loop correctly rejects it and starts retrying — but the SSE layer couldn't tell that per-tool retry apart from a genuinely fatal failure, so it ended the response and cancelled the still-retrying agent, dead-ending the turn with no answer and no stats line. Reproduced 4/4 on-hardware before this fix. Now a recoverable error surfaces as a non-terminal status line (the user still sees the failure) and the retry can reach completion.Closes #2515
Test plan
python -m pytest tests/unit -k "sse or agent_error" -q— 399 passedpython -m pytest hub/agents/email/python/tests -k "sse or translation or query_routes" -q— 64 passed/v1/email/queryroute + realSSEOutputHandler+ real translation layer with a fake agent that emits a recoverable per-tool error then retries: assertsrun.cancel_eventandhandler.cancelledstay unset mid-run and the stream reachesfinalwith both the failed and retried tool calls streamed (test_recoverable_tool_error_does_not_terminate_stream_or_cancel_run)python util/lint.py --all— all blocking checks pass