[SDK] Enhance timeout controls + reporting#1586
Conversation
dosubot
bot
added
the
size:XXL
✅ Snyk checks have passed. No issues have been found so far.
💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse. |
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
| toolName: toolCall.toolName, | ||
| arguments: (toolCall.input ?? {}) as Record<string, unknown>, | ||
| })) | ||
| ); |
There was a problem hiding this comment.
Unguarded stepResult.toolCalls access may crash on unexpected shape
Low Severity
The onStepFinish callback accesses stepResult.toolCalls.map(...) without a defensive null/array check. The existing extractToolCalls in tool-extraction.ts defensively guards the same property with step.toolCalls && Array.isArray(step.toolCalls). If the AI SDK ever delivers a step result with a missing or non-array toolCalls (e.g. during an abort or error mid-step), this would throw a TypeError inside the generateText call, potentially masking the real error and losing partial results that the PR is specifically trying to preserve.
1 participant
Note
Medium Risk
Changes core eval runtime behavior (timeouts/abort propagation and tool execution short-circuiting), which could affect test outcomes and reporting semantics, but is scoped to the SDK and covered by new unit tests.
Overview
Enhances SDK timeout + early-exit controls for evals.
PromptOptionsnow supportsabortSignal,timeoutMs, andstopAfterToolCall(stop after a named tool is selected, stub its result, and still capture tool name/args).Improves timeout behavior and reporting.
EvalTestiterations now enforce timeouts by aborting the active prompt and allowing a 1s grace period before hard-failing, while aggregating whatever prompt history, tokens, and latencies were captured.TestAgentnow preserves partial messages/tool calls/token usage when prompts are aborted/timeout, and skips MCP widget snapshot capture for short-circuited tools.Docs and
sdk/README.mdare updated to describe the new options and clarifytimeoutMssemantics; tests add coverage for partial-timeout success, multi-turn metric preservation, abort handling,stopAfterToolCall, and the new timeout/abort passthroughs.Written by Cursor Bugbot for commit 6f82805. This will update automatically on new commits. Configure here.