-
Notifications
You must be signed in to change notification settings - Fork 521
Feat/preserve thought signature for gemini3 #1227
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Feat/preserve thought signature for gemini3 #1227
Conversation
- Add thoughtSignature field to ToolUse TypedDict as optional field - Add thoughtSignature to ContentBlockStartToolUse TypedDict - Preserve thoughtSignature during streaming event processing - Fixes compatibility with Gemini 3 Pro thinking mode This change enables proper multi-turn function calling with Gemini 3 Pro, which requires thought_signature to be passed back in subsequent requests. Resolves: Gemini 3 Pro 400 error for missing thought_signature See: https://ai.google.dev/gemini-api/docs/thought-signatures
- Capture thought_signature from Gemini function call responses - Base64 encode thought_signature for storage in message history - Decode and pass thought_signature back to Gemini in subsequent requests - Configure thinking_config to disable thinking text but preserve signatures - Add NotRequired import to content.py for type safety This complements the framework changes by implementing Gemini-specific handling of thought signatures for proper multi-turn function calling with Gemini 3 Pro. See: https://ai.google.dev/gemini-api/docs/thought-signatures
- Fix variable name conflict with thought_signature - Break long lines to comply with 120 character limit - Use explicit type annotations for thought signature variables
ed1837d to
a20b7ea
Compare
|
@dpolistwm I see that you have created two new test files. I dont recommend that, the tools test file already exists here and creating new test files for testing small logic is something the maintainers wouldn't like I think. Make sure you only add tests for the fix you are proposing. The best option I think here would be just waiting for a maintainer to see what approach they would take. Thanks. |
- Rename test_tools.py -> test_tool_use.py (tests ToolUse TypedDict) - Rename test_content.py -> test_content_block_start_tool_use.py (tests ContentBlockStartToolUse TypedDict) This makes the test file names more descriptive and avoids confusion with tests/strands/tools/test_tools.py
|
Hey @mkmeral , I understand your concern, but implementing this workaround and degrading the tought process of such a popular model would make strands agents a "second class citizen" among other frameworks in the market. Many users may stop using Strands Agents with Gemini 3 Pro (one of the most popular, fast and powerful LLMs in the market) if this is the official solution and probably choose ADK instead. While it does not support AWS Agentcore directly (memory, etc.), it does support Google's equally feature-rich AgentEngine counterpart. We would certainly consider that. Please take this risk into account when making your decision. |
|
@dpolistwm actually I think I found a way to get the feature working with your PR, and leave the tool use event type as is. Gemini 3 Pro only validates the latest thought signature, not all of them. So what we can do instead is keep track of the thought signatures in Can you update your PR following this approach? One other thing to note: we would also still want to apply Does this make sense? |
|
I'll try and make this work ;) |
This refactors the thoughtSignature feature to be self-contained within the GeminiModel class instead of spreading it across the SDK: Changes: - Removed thoughtSignature from ContentBlockStartToolUse (content.py) - Removed thoughtSignature from ToolUse (tools.py) - Removed thoughtSignature handling from streaming.py - Added self.last_thought_signature instance attribute to GeminiModel - GeminiModel now stores thought_signature from responses and uses it for subsequent requests automatically Benefits: - Cleaner SDK types without Gemini-specific fields - Simpler implementation without base64 encoding/decoding in messages - thought_signature is automatically tracked per model instance - No changes required to SDK-level streaming pipeline Tests updated accordingly to verify internal storage behavior.
…ini.py Reverted content.py, tools.py, streaming.py and test files to their original state. thoughtSignature handling is now self-contained in GeminiModel using self.last_thought_signature.
|
Now the only changed file in this PR is gemini.py with the last thought signature. I tested it locally against a real project and it is working fine 💯 |
|
Working on "0 tools" implementation (getting a 400 error)... |
|
Looks fine now to me. Please let me know if you need anything else |
|
Hello @mkmeral, could my PR also be used in some way, since @dpolistwm mentions gemini 3 doesnt work in my PR, maybe we can figure out a way to fix it. Or if this PR is already correct, I will close my PR then ig. |
src/strands/models/gemini.py
Outdated
| Returns: | ||
| Gemini request config. | ||
| """ | ||
| # Disable thinking text output when tools are present |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why? Currently developers can configure this right? Do we need to explicitly set it for everyone?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Makes perfect sense.
|
|
||
| # Store the last thought_signature from Gemini responses for multi-turn conversations | ||
| # See: https://ai.google.dev/gemini-api/docs/thought-signatures | ||
| self.last_thought_signature: Optional[bytes] = None |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
can we default to skip? This way, if developers change model provider (e.g. anthropic to gemini), the agent would still work
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is contained within the Gemini model implementation so it doesn't affect other providers. Could you please ellaborate? (the absence of thought signatures is also the root cause of the current incompatibility with Gemini 3 Pro)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actually 2 use cases:
First, customers can switch model providers. So imagine they start with Bedrock, and then they want to change the agent's model provider to gemini 3
# Written is semi-psuedocode, don't focus on code too much 😅
agent = Agent(model=BedrockModel(), tools=[tool1, tool2])
# agent is being used with tools
agent(...)
# later (for some reason) devs might try to update the model providers to Gemini 3
agent.model = GeminiModel(modelId="3-pro")
# agent is being invoked. At this point, it has the original message history with tool use, but it has a clean gemini model provider. If we do not skip the
agent(...)
Second, session managers can be problematic. If agent's context is loaded from session managers, the thought signature will not exist. We need to ensure for these cases, we do not throw exception.
# Create a session manager with a unique session ID
session_manager = FileSessionManager(session_id="test-session")
agent = Agent(session_manager=session_manager)
agent("Hello!") # This conversation is persisted
# Couple hours later, in another runtime instance, running the same code again
session_manager = FileSessionManager(session_id="test-session") # Loads the messages from file system
agent = Agent(session_manager=session_manager)
agent("Hello!") # Throws error due to though signatures
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
From what I understood reading Google's docs, the requirement for the preservation of tought signatures needs to happen within a turn/cycle of the agent loop. So, if a session/memory was created using another model, during the next turn execution with Gemini 3, a first/new thought signature would be introduced and be used. I can try to run both scenarios to try and reproduce.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Here's a quick script I've used:
#!/usr/bin/env python3
"""Example script using strands agents with FileSystem session manager."""
import os
from uuid import uuid4
from dotenv import load_dotenv
load_dotenv()
from strands import Agent, tool
from strands.models.gemini import GeminiModel
from strands.session import FileSessionManager
from strands.models.bedrock import BedrockModel
@tool
def get_weather(city: str) -> str:
return f"The weather in {city} is sunny."
@tool
def greet_user(user_name: str) -> str:
return f"Hello, {user_name}! How can I help you today?"
def main():
"""Main function demonstrating strands agents with FileSessionManager."""
# Create a storage directory for sessions
storage_dir = os.path.join(os.getcwd(), "sessions")
# Generate a session ID (or use a fixed one for persistence)
session_id = str(uuid4())
print(f"Session ID: {session_id}")
# Create FileSessionManager
session_manager = FileSessionManager(
session_id=session_id,
storage_dir=storage_dir
)
# Create GeminiModel with gemini-3-pro-preview
gemini_model = GeminiModel(
model_id="gemini-3-pro-preview",
)
bedrock_model = BedrockModel(
model_id="us.anthropic.claude-3-7-sonnet-20250219-v1:0",
)
# Create an agent with the session manager and Bedrock model
agent = Agent(
system_prompt="You are a helpful assistant that can answer questions and help with tasks. When the user provides their name, greet them using the greet_user tool.",
model=bedrock_model,
tools=[greet_user, get_weather],
session_manager=session_manager,
agent_id="my_agent"
)
agent("Hello! My name is Daniel.")
agent.model = gemini_model
agent ("What is the weather in Tokyo?")
if __name__ == "__main__":
main()Worked fine :) Here's the output:
➜ test-gemini3 uv run python main.py
Session ID: 8721956e-28db-4ee0-abe1-68a39ad64d08
Tool #1: greet_user
Hello Daniel! It's nice to meet you. How can I assist you today? I'm here to help answer questions or assist with tasks you might have.
Tool #2: get_weather
The weather in Tokyo is currently sunny
| id=content["toolUse"]["toolUseId"], | ||
| name=content["toolUse"]["name"], | ||
| ), | ||
| thought_signature=self.last_thought_signature, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This would apply it to all tool uses right? Is there a way we can make it more specific?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is required per Gemini docs:
"As a general rule, if you receive a thought signature in a model response, you should pass it back exactly as received when sending the conversation history in the next turn. When using Gemini 3 Pro, you must pass back thought signatures during function calling, otherwise you will get a validation error (4xx status code)."
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I meant more of this line, where we don't actually need to set this for all tool calls, just the first one. That said, this is more of a nit comment
The first functionCall part in each step of the current turn must include its thought_signature
dpolistwm
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Removed hard-coded though configuration
Description
This PR adds support for preserving Gemini's
thoughtSignaturefield during function calling, which is required by Gemini 3 Pro for multi-turn conversations with tools.Problem
When using Gemini 3 Pro with function calling, the model returns a
thought_signaturefield that must be passed back in subsequent requests. Currently, Strands drops this field during streaming and message reconstruction, causing 400 errors:Unable to submit request because function call tool_name in the 2. content block is missing a thought_signature
This affects multi-turn conversations, nested agents, and any workflow requiring multiple function call exchanges with Gemini 3 Pro.
Solution
Framework Changes:
src/strands/types/tools.py: AddedthoughtSignature: NotRequired[str]toToolUseTypedDict (changed tototal=Falseto support optional field)src/strands/types/content.py: AddedthoughtSignature: NotRequired[str]toContentBlockStartToolUseTypedDict and addedNotRequiredimportsrc/strands/event_loop/streaming.py: PreservethoughtSignaturewhen processing streaming chunks (2 locations: extracting from tool use data and passing through to ToolUse object creation)Gemini Provider Changes:
4.
src/strands/models/gemini.py:thought_signaturefrom Gemini function call responsesthinking_configto disable thinking text while preserving signaturesTechnical Details
The
thoughtSignatureis an encrypted token provided by Gemini that preserves the model's reasoning context. It's:NotRequired[str]) that doesn't affect other providersReference: Gemini Thought Signatures Documentation
Related Issues
None - this is a new feature to support Gemini 3 Pro's function calling requirements.
Documentation PR
N/A - No documentation changes needed (inline code comments added for clarity)
Type of Change
New feature
Testing
How have you tested the change?
Tested with:
gemini-3-pro-preview) with function callingResults:
thought_signatureVerified in consuming repositories:
Note on CI: There is 1 pre-existing mypy error in
src/strands/tools/tools.py:47(unused type: ignore comment) that is unrelated to these changes. All type checking passes for the files modified in this PR.hatch run prepareChecklist
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.
Related Issues
Fixes #1199