.Net: Bug: Gemini Connector does not support BinaryContent bringing it inline with other Connectors #13131
Description
Describe the bug
The Gemini Connector only supports TextContent, ImageContent, and AudioContent KernelContent types. Other connectors now support the BinaryContent type that the Chat Completion API supports allowing you to upload other file types encoded as a base64 string.
The Gemini connector actually already supports arbitrary base64 PDF content by prioritising binary data in the CreateGeminiPartFromImage function in the GeminiRequest.cs. This is somewhat weird as to send PDFs you have to use ImageContent which is counter-intuitive and different to how the other connectors handle this.
I suggest we keep the current behaviour but also add explicit support for the BinaryContent KernelContent type in the GetGeminiPartFromKernelContent function.
To Reproduce
string PdfBase64Data = "JVBERi0xLjQKMSAwIG9iago8PC9UeXBlIC9DYXRhbG9nCi9QYWdlcyAyIDAgUgo+PgplbmRvYmoKMiAwIG9iago8PC9UeXBlIC9QYWdlcwovS2lkcyBbMyAwIFJdCi9Db3VudCAxCj4+CmVuZG9iagozIDAgb2JqCjw8L1R5cGUgL1BhZ2UKL1BhcmVudCAyIDAgUgovTWVkaWFCb3ggWzAgMCA1OTUgODQyXQovQ29udGVudHMgNSAwIFIKL1Jlc291cmNlcyA8PC9Qcm9jU2V0IFsvUERGIC9UZXh0XQovRm9udCA8PC9GMSA0IDAgUj4+Cj4+Cj4+CmVuZG9iago0IDAgb2JqCjw8L1R5cGUgL0ZvbnQKL1N1YnR5cGUgL1R5cGUxCi9OYW1lIC9GMQovQmFzZUZvbnQgL0hlbHZldGljYQovRW5jb2RpbmcgL01hY1JvbWFuRW5jb2RpbmcKPj4KZW5kb2JqCjUgMCBvYmoKPDwvTGVuZ3RoIDUzCj4+CnN0cmVhbQpCVAovRjEgMjAgVGYKMjIwIDQwMCBUZAooRHVtbXkgUERGKSBUagpFVAplbmRzdHJlYW0KZW5kb2JqCnhyZWYKMCA2CjAwMDAwMDAwMDAgNjU1MzUgZgowMDAwMDAwMDA5IDAwMDAwIG4KMDAwMDAwMDA2MyAwMDAwMCBuCjAwMDAwMDAxMjQgMDAwMDAgbgowMDAwMDAwMjc3IDAwMDAwIG4KMDAwMDAwMDM5MiAwMDAwMCBuCnRyYWlsZXIKPDwvU2l6ZSA2Ci9Sb290IDEgMCBSCj4+CnN0YXJ0eHJlZgo0OTUKJSVFT0YK";
var chatService = new GoogleAIGeminiChatCompletionService(modelId: "gemini-2.5-flash", apiKey: "my-key");
var chatHistory = new ChatHistory("Summarise any content provided to you into 5 words");
chatHistory.AddUserMessage([
new TextContent("File #1"),
new BinaryContent(PdfBase64Data)
]);
var response = await chatService.GetChatMessageContentAsync(chatHistory);
// Error: Unsupported content type. BinaryContent is not supported by Gemini.Expected behaviour
BinaryContent is accepted as a valid message type and supports uploading of base64 PDFs just like it currently does with the ImageContent.
Platform
- Language: C#
- Connector: Gemini
Additional context
I'm happy to attempt a PR for this myself, I just wanted to follow due process first.
Metadata
Metadata
Assignees
Type
Projects
Status