ComputerUseToolset: safety_decision for click_at requires safety_acknowledgement even after user confirms in ADK UI

[anant-acog]

Description of the bug:

When using the ComputerUseToolset sample with gemini-2.5-computer-use-preview-10-2025, I get a 400 INVALID_ARGUMENT error from the Gemini API after confirming a safety prompt in the ADK web UI.

The error:

400 INVALID_ARGUMENT. {
"error": {
"code": 400,
"message": "The safety decision from function call click_at must be acknowledged in the corresponding function response.",
"status": "INVALID_ARGUMENT"
}
}

From the message, it looks like the tool response that ADK sends back to Gemini is missing the required safety_acknowledgement field for a click_at call that has a safety_decision.

I am not running my own model loop – I’m using the standard ADK Agent + ComputerUseToolset + adk web UX. I do click “Yes” when the safety confirmation dialog appears in the UI, but the next call to generate_content still fails with the error above.

Environment / Versions

OS: macOS (Apple Silicon)

Python: 3.13.9

google-adk: 1.20.0

google-genai: >=1.52.0

playwright: 1.56.0

browserbase: >=1.4.0 (installed, but not actively using Browserbase for this repro)

rich: >=14.2.0

termcolor: 3.2.0

Model: gemini-2.5-computer-use-preview-10-2025

Dependency block:

dependencies = [
"browserbase>=1.4.0", # only if you actually use Browserbase
"google-adk==1.20.0",
"google-genai>=1.52.0",
"playwright==1.56.0",
"rich>=14.2.0",
"termcolor==3.2.0",
]

Code / Setup

I’m using the sample Computer Use project from the ADK repo:

https://github.com/google/adk-python/tree/main/contributing/samples/computer_use

With a minimal agent that looks like this:

agent.py

from google.adk import Agent
from google.adk.models.google_llm import Gemini
from google.adk.tools.computer_use.computer_use_toolset import ComputerUseToolset
from typing_extensions import override

from .playwright import PlaywrightComputer

root_agent = Agent(
model="gemini-2.5-computer-use-preview-10-2025",
name="hello_world_agent",
description=(
"computer use agent that can operate a browser on a computer to finish user tasks"
),
instruction="you are a computer use agent",
tools=[
ComputerUseToolset(
computer=PlaywrightComputer(screen_size=(800, 600))
)
],
)

PlaywrightComputer is basically the same as the sample in contributing/samples/computer_use/playwright.py (Chromium via Playwright, returns ComputerState(screenshot, url)). I haven’t added any custom LLM loop logic – I let ADK run everything via adk web.

Steps to Reproduce

Clone the ADK repo and/or the contributing/samples/computer_use sample, or set up the equivalent project with the Agent above.

Install dependencies:

pip install google-adk==1.20.0 google-genai>=1.52.0 playwright==1.56.0 browserbase>=1.4.0 rich>=14.2.0 termcolor==3.2.0
playwright install chromium

Run the ADK dev server (from the sample project):

adk web .

Open the web UI in the browser and talk to the hello_world_agent.
For example, ask:

“Go to biohaven.com and go to pipeline page and grab information.”

At some point the model issues a click_at with a safety_decision, and the ADK UI shows a safety confirmation dialog.

Click “Yes” in the UI to allow the action.

Observe in the backend logs:

google.genai.errors.ClientError: 400 INVALID_ARGUMENT. {
"error": {
"code": 400,
"message": "The safety decision from function call click_at must be acknowledged in the corresponding function response.",
"status": "INVALID_ARGUMENT"
}
}

The stack trace points back into:

google/adk/models/google_llm.py::generate_content_async

google/genai/_api_client.py

with the 400 ClientError from models.generate_content(...).

Actual vs expected behavior:

No response

Any other information you'd like to share?

No response