.Net: Bug: SK can't call AzureOpenAI o1/o3-mini models

[rwjdk]

Describe the bug
When doing an agent.InvokeAsync call against the o1 model (version:2024-12-17) in Azure Open AI service you get error:

"HTTP 400 (BadRequest)\r\n\r\nModel o1 is enabled only for api versions 2024-12-01-preview and later"

To Reproduce
Steps to reproduce the behavior:

1. Make a kernel chat completion service that target o1 as deployment model
2. Make an agent (with instructions, temperature and structured output)
3. Call agent.InvokeAsync

Expected behavior
A normal LLM respose (in this case json due to structured output)

Platform

• OS: Windows
• IDE: Visual Studio
• Language: C#
• Source: NuGet package version 1.33.0

Additional context
Issue is properly the transative use of the azure.ai.openai beta2 package. I tried manually bumping this to latest, but it gave an feature not implemented exception

[rwjdk]

Relate to Azure/azure-sdk-for-net#47809

[moonbox3]

Hi @rwjdk, can you please make sure you are indeed sending the 2024-12-01-preview Azure OpenAI API version. Based on this error message, your Azure api version looks to be older than 2024-12-01-preview. The o1 model (2024-12-17) is the model release date and is not the same as the api version.

[rwjdk]

@moonbox3: As I'm using AzureOpenAI services I'm not sending a version at any point in the call (just the deployment name which in my case is "o1") I've checked the raw request to make sure and no version is mentioned in there

In Azure AI Studio the version is 2024-12-17 (the only option)

But taking above image I saw something odd - The target URI that you do not choose yourself (and never use in SK says api-version=2024-12-01-preview so it could be an Azure bug [I'm in swedenCentral Azure Region])

Can one in SK alter the target URI version?

[rwjdk]

Did a quick test with the Azure.AI.OpenAI (v2.1.0) nuget package directly and it gives the same error so I guess SK Team depends on having this dependency supporting/working first :-/

[moonbox3]

Yes, you can use a different Azure API version. Tagging @SergeyMenshykh for a specific example on how to in .Net.

[rwjdk]

@moonbox3 I tried that but as the underlying platform do not yet support the version it still fails with a not implemented exception :-/

Assume this is what you mean:
kernelBuilder.AddAzureOpenAIChatCompletion(deploymentModel, Endpoint, key, apiVersion: "some-version here");

[SergeyMenshykh]

It's not supported by the Azure.AI.OpenAI SDK yet - https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/openai/Azure.AI.OpenAI/src/Custom/AzureOpenAIClientOptions.cs
cc: @RogerBarreto

[moonbox3]

Apologies for misleading @rwjdk. The ability to specify a custom API version is allowed in SK Python, but per @SergeyMenshykh, this isn't supported in the Azure.AI.OpenAI SDK yet.

[RogerBarreto]

As of now, the SDK doesn't have an option for 2024-12-01-preview in their main branch yet.

https://github.com/Azure/azure-sdk-for-net/blob/a0e7401307f093ce84a66d572cd5d43692293176/sdk/openai/Azure.AI.OpenAI/src/Custom/AzureOpenAIClientOptions.cs#L78

Blocking this as this depends on Azure SDK to support the version.

Tracking issue:

• [FEATURE REQ] add 2024-12-01-preview apiVersion in AzureOpenAIClientOptions Azure/azure-sdk-for-net#47809

[RogerBarreto]

As a workaround you can add a Handler to your connectors thru a httpClient. I managed to override both version and max_completion_token limitations in the current SDK.

Important

This is a breaking glass scenario and should be dropped as soon the Azure OpenAI SDK supports o1 models and most API versions (like 2025-01-01, 2024-12-01).

Usage

var overrideApiVersion = "2025-01-01-preview";
using var httpClient = new HttpClient(new AzureOverrideHandler(overrideApiVersion));
var apiKey = config["AzureOpenAI:ApiKey"]!;
var endpoint = config["AzureOpenAI:Endpoint"]!;

var kernel = Kernel.CreateBuilder()
    .AddAzureOpenAIChatCompletion("o1-mini", endpoint, apiKey, httpClient: httpClient)
    .Build();

Http Handler (For version override, and max token count fix)

public partial class AzureOverrideHandler: HttpClientHandler
{
    private string? _overrideApiVersion;
    public AzureOverrideHandler(string? overrideApiVersion = null)
    {
        this._overrideApiVersion = overrideApiVersion;
    }
    protected override async Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
    {
        var options = new JsonSerializerOptions()
        {
            WriteIndented = true
        };

        using var oldContent = request.Content;
        if (oldContent is not null && request.RequestUri is not null)
        {
            var requestBody = await oldContent.ReadAsStringAsync(cancellationToken);
            if (requestBody.IndexOf("\"model\":\"o1") > 0 && requestBody.IndexOf("\"max_tokens\":") > 0)
            {
                requestBody = requestBody.Replace("\"max_tokens\":", "\"max_completion_tokens\":");
                request.Content = new StringContent(requestBody, new MediaTypeHeaderValue("application/json"));
            }

            // Console.WriteLine("Request body: " + JsonSerializer.Serialize(JsonSerializer.Deserialize<JsonElement>(requestBody), options));
        }

        if (this._overrideApiVersion is not null && request.RequestUri is not null)
        {
            var requestUri = request.RequestUri.ToString();
            var currentVersion = CurrentApiVersionRegex().Match(requestUri).Value;

            if (!string.IsNullOrEmpty(currentVersion))
            {
                request.RequestUri = new Uri(requestUri.Replace(currentVersion, this._overrideApiVersion));
            }
            else
            {
                request.RequestUri = new Uri($"{requestUri}?api-version={this._overrideApiVersion}");
            }

            // Console.WriteLine(request.RequestUri);
        }

        return await base.SendAsync(request, cancellationToken);
    }

    [GeneratedRegex(@"\d{4}-\d{2}-\d{2}(-preview)?$")]
    public static partial Regex CurrentApiVersionRegex(); 
}

[rwjdk]

Cool. Thank you @RogerBarreto . Will try it out first thing tomorrow

[rwjdk]

@RogerBarreto: as a followup for your nice workaround (sorry for the delay in answer)... just some notes on what works and does not work so far using SK against the o1 model (with the break-glass approach). No blockers personally for me and properly on the Ai.Azure.OpenAi Nuget teams side but just FYI

Works:

• Chats 👍
• Agents 👍
• Function Calling 👍
• Structured Output 👍

Does not work:

• Setting Temperature to anything other than 1
• IAutoFunctionInvocationFilter is not called for some odd reason on o1 when using Function Calling (perhaps it is the break-glass thing shortcutting it)

[rwjdk]

FYI: This also work with o3-mini 👍 with same limitations (+ the option to set o3-mini's thinking mode "low", "medium" or "high")

[dsfsdsdsfsds]

@rwjdk is there a way to set thinking mode in the solution provided by @RogerBarreto for both o1 and o3-mini?

[rwjdk]

@dsfsdsdsfsds Not as far as I can see... it wors but one would guess medium is the default used

[RogerBarreto]

@rwjdk Thanks for the feedback.

Regarding the temperature clearly the API is giving an error where states the model doesn't support usage of different temperatures, only the default (1). Not much to do here as this is seems to be a backend limitation.

Unsupported value: 'temperature' does not support 0.1 with this model. Only the default (1) value is supported.

Attempt to use tools with o1-mini also gives an error as tools is not supported.

Unsupported parameter: 'tools' is not supported with this model.'

Now running the plugins against the o3-mini the result was expected and the function calling triggered the function and the filter.

Here's the update Use Case code how I managed to do it:

var builder = Kernel.CreateBuilder()
    .AddAzureOpenAIChatCompletion("o1-mini", endpoint, apiKey, httpClient: httpClient);

// Ensure you add the filter using the `IAutoFunctionInvocationFilter` interface into the `IServiceCollection`.
builder.Services.AddSingleton<IAutoFunctionInvocationFilter>(new MyAutoFunctionInvocationFilter());

var myFunction = KernelFunctionFactory.CreateFromMethod(() => DateTime.Now.ToString("G"), "CurrentDate");
builder.Plugins.Add(KernelPluginFactory.CreateFromFunctions("myPlugin", [myFunction]));

var kernel = builder.Build();
var chatService = kernel.GetRequiredService<IChatCompletionService>();


var response = await chatService.GetChatMessageContentAsync("What is the current date?", 
    new OpenAIPromptExecutionSettings { 
        FunctionChoiceBehavior = FunctionChoiceBehavior.Auto(), // Enable function calling feature
        MaxTokens = 1000, 
        Temperature = 1 
    }, 
    kernel); // Is important to provide the kernel for the functions to be provided to the AI Model.

public class MyAutoFunctionInvocationFilter : IAutoFunctionInvocationFilter
{
    public async Task OnAutoFunctionInvocationAsync(AutoFunctionInvocationContext context, Func<AutoFunctionInvocationContext, Task> next)
    {
        Console.WriteLine("Before function invocation");
        await next(context);
        Console.WriteLine("After function invocation");
    }
}

[rwjdk]

New issue opened on the Azure SDK side: Azure/azure-sdk-for-net#48110 regarding o3-mini

[HillPhelmuth]

@rwjdk is there a way to set thinking mode in the solution provided by @RogerBarreto for both o1 and o3-mini?

Yes, you can set the thinking mode with this change:

var requestBody = await oldContent.ReadAsStringAsync(cancellationToken);
if (requestBody.IndexOf("\"model\":\"o1") > 0 && requestBody.IndexOf("\"max_tokens\":") > 0)
{
    requestBody = requestBody.Replace("\"max_tokens\":", "\"max_completion_tokens\":");
    var root = JsonNode.Parse(requestBody); // Parse 
    root["reasoning_effort"] = "high"; // add reasoning effort
    var modifiedJson = root.ToJsonString(new JsonSerializerOptions
    {
        WriteIndented = true
    }); // serialize back to string
   request.Content = new StringContent(modifiedJson, new MediaTypeHeaderValue("application/json"));
}