fix: resolve schema conversion errors for Gemini in evaluator optimizer #215
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This pull request addresses errors encountered with the evaluator optimizer pattern when using Google Gemini models. The primary issue was related to the conversion and handling of Pydantic model schemas for the Google Generative AI API.
Changes Made:
src/mcp_agent/llm/providers/augmented_llm_google_native.py
:response_schema
.schema
is provided, it's now explicitly converted to a Google Schema object usingself._converter.json_schema_to_google_schema
. This ensures that the schema structure is compatible with what the Google API expects for structured output.schema
is provided, it falls back to inferring the schema using_get_schema_type(model)
.src/mcp_agent/llm/providers/google_converter.py
:json_schema_to_google_schema
: This method was significantly improved to robustly convert Pydantic-generated JSON schemas togoogle.genai.types.Schema
.$ref
s within the JSON schema, resolving them against theroot_schema
.type: null
,type: ["string", "null"]
) to map them correctly togoogle.genai.types.Type
and thenullable
attribute.properties
(for objects) anditems
(for arrays).title
as a fallback fordescription
if the latter is missing.enum
to ensure only string enums are passed as Google schema enums._clean_schema_for_google
: Removed some potentially overly aggressive cleaning of schema keywords to preserve necessary information liketitle
.FunctionResponse
and media parts.Impact:
These changes ensure that Pydantic models used in tool calls and for structured output with Gemini models are correctly converted to the Google API's schema format. This resolves previous errors related to schema validation and should lead to more reliable behavior of the evaluator optimizer pattern and other features relying on structured data exchange with Gemini.