Skip to content

Conversation

@bendrucker
Copy link
Contributor

@bendrucker bendrucker commented Dec 19, 2025

Adds tool approval integration for the Vercel AI adapter, enabling human-in-the-loop workflows with AI SDK UI.

Summary

  • Add opt-in tool_approval flag for simplified tool approval workflows
  • Emit tool-approval-request chunks for deferred tool approvals
  • Emit tool-output-denied chunks when user denies tool execution
  • Auto-extract approval responses from follow-up requests
  • Add approval field to tool parts with ToolApprovalRequested/ToolApprovalResponded states

Usage

from pydantic_ai import Agent
from pydantic_ai.tools import DeferredToolRequests
from pydantic_ai.ui.vercel_ai import VercelAIAdapter

agent: Agent[None, str | DeferredToolRequests] = Agent(
    'openai:gpt-5',
    output_type=[str, DeferredToolRequests],
)

@agent.tool_plain(requires_approval=True)
def delete_file(path: str) -> str:
    """Delete a file from the filesystem."""
    os.remove(path)
    return f'Deleted {path}'

@app.post('/chat')
async def chat(request: Request) -> Response:
    adapter = await VercelAIAdapter.from_request(request, agent=agent, tool_approval=True)
    return adapter.streaming_response(adapter.run_stream())

When tool_approval=True, the adapter will:

  1. Emit tool-approval-request chunks when tools with requires_approval=True are called
  2. Automatically extract approval responses from follow-up requests
  3. Emit tool-output-denied chunks for rejected tools

AI SDK Tool Approval Protocol

Tool approval is an AI SDK v6 feature (not available in v5, the current stable). Here's how the protocol works:

Protocol Flow

sequenceDiagram
    participant Model
    participant Server
    participant Client

    Model->>Server: tool call
    Server->>Client: tool-input-start
    Server->>Client: tool-input-available
    Server->>Client: tool-approval-request

    Note over Client: User approves/denies

    Client->>Server: approval response (next request)

    alt Approved
        Server->>Server: Execute tool
        Server->>Client: tool-output-available
    else Denied
        Server->>Client: tool-output-denied
        Server->>Model: denial info
    end
Loading

Chunk Types

tool-approval-request

Server → client:

{
  "type": "tool-approval-request",
  "approvalId": "<uuid>",
  "toolCallId": "<tool-call-id>"
}

tool-output-denied

Server → client, when denied:

{
  "type": "tool-output-denied",
  "toolCallId": "<tool-call-id>"
}

Why the Server Emits This Chunk

"Pydantic AI doesn't have an event for denials because denials never originate in Pydantic AI itself, they come from the user in the form of DeferredToolResults passed into agent.run" — #3760 comment

This is correct—the denial decision originates from the user via the ToolApprovalResponded field in tool parts. However, the AI SDK protocol expects the server to emit tool-output-denied for two reasons:

  1. The client needs confirmation that the tool lifecycle is complete. When the server receives a denial and emits tool-output-denied, the client can transition its UI from "awaiting result" to "denied".

  2. Just as the server emits tool-output-available when a tool executes successfully, it emits tool-output-denied when execution is skipped due to denial. This gives the client a consistent signal for each tool call's final state.

The flow is:

  1. Client sends denial via approval: { id, approved: false, reason } in the tool part
  2. Server extracts this from deferred_tool_results and passes it to the agent
  3. When processing the denied tool result, server emits tool-output-denied to the client
  4. Client updates UI to show the tool was denied

Tool Part Approval States

Tool parts include an approval field tracking the approval lifecycle:

Awaiting Response

{ "id": "<approval-id>" }

User Responded

{ "id": "<approval-id>", "approved": true/false, "reason": "optional" }

Two-Step Flow

Unlike regular tool calls, approved tools require two model interactions:

  1. Model requests tool → server returns with tool-approval-request → awaits user
  2. User decision sent → server executes (if approved) or informs model (if denied)

Changes

Types

  • Add ToolApprovalRequested and ToolApprovalResponded types
  • Add ToolApprovalRequestChunk and ToolOutputDeniedChunk response chunks
  • Add approval field to all ToolUIPart and DynamicToolUIPart variants

Event Stream

  • Emit ToolApprovalRequestChunk when agent returns DeferredToolRequests (only when tool_approval=True)
  • Emit ToolOutputDeniedChunk when processing denied tool results (only when tool_approval=True)
  • Add denied_tool_ids property to track which tools were denied

Adapter

  • Add tool_approval parameter to from_request() for opt-in behavior
  • Add deferred_tool_results instance field on base UIAdapter class
  • Auto-extract approval responses when tool_approval=True
  • Auto-pass deferred_tool_results to agent via instance field fallback

Documentation

  • Add Tool Approval section with simplified usage example
  • Link to Pydantic AI deferred tools documentation

Testing

  • Test ToolApprovalRequestChunk emission when requires_approval=True tool is called
  • Test ToolOutputDeniedChunk emission when user denies tool execution
  • Test approval chunks are NOT emitted when tool_approval=False
  • Test from_request() correctly handles tool_approval parameter
  • Test deferred_tool_results fallback from instance field
  • Test denied_tool_ids extraction for both dynamic and builtin tools
  • Test extract_deferred_tool_results() for approved, denied, and no-approval cases
  • Snapshot updates for new approval field on tool parts

References

@bendrucker bendrucker force-pushed the vercel-ai-tool-approval branch from 099f07a to 1160591 Compare December 19, 2025 05:03
@bendrucker bendrucker marked this pull request as ready for review December 19, 2025 05:27
@bendrucker
Copy link
Contributor Author

Thanks for the quick reviews! Will get the test issue fixed and reply to your comments shortly.

- Add test for from_request() with tool_approval parameter

- Add test verifying approval chunks not emitted when tool_approval=False

- Add test for deferred_tool_results fallback from instance field

- Add test for denied_tool_ids with ToolUIPart (builtin tools)

- Fix docs link to point to VercelAIAdapter.from_request

- Reword Tool Approval section to clarify AI SDK UI vs AI Elements
- Make _extract_deferred_tool_results private and inline into from_request
- Make _denied_tool_ids a private cached_property
- Simplify approval values to True/False (per reviewer suggestion)
- Add AI SDK v6 requirement note to documentation and docstrings
- Update tests for new API and simplified return values
- Inline extract_deferred_tool_results logic into from_request()
- Remove unit tests for private _extract_deferred_tool_results method
- Remove unit tests for private _denied_tool_ids property
- Update test_tool_output_denied_chunk_emission to use public interface
- Update test_tool_output_denied_chunk_emission to use from_request()
  with explicit type binding to test the full public interface
- Remove test_from_request_with_tool_approval_enabled (now redundant)
- Remove test_deferred_tool_results_fallback_from_instance (tested
  internal plumbing rather than observable behavior)
@bendrucker bendrucker force-pushed the vercel-ai-tool-approval branch from 52dea3e to 676530b Compare December 20, 2025 09:52
@bendrucker bendrucker force-pushed the vercel-ai-tool-approval branch from 2deeb25 to 9eebd80 Compare December 20, 2025 11:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants