Support reasoning summary models in AzureOpenAIEvalClient

[taniokay]

LiteLLMEvalClient already supports Responses API, but litellm.responses is still in beta, so this PR supports Responses API in AzureOpenAIEvalClient too.

---

Note

Adds Responses API-based reasoning summary to OpenAI and Azure eval clients, adjusts outputs/logprobs behavior, improves error logging, and bumps version.

• Eval Clients (OpenAI/Azure):
  • Add optional reasoning summary support (use_reasoning_summary, reasoning_effort, reasoning_summary).
  • Route requests via new _dispatch to switch between Chat Completions and Responses API.
  • In get_text_responses, extract content and append reasoning summaries when enabled.
  • Disallow log-likelihood retrieval when reasoning summary is enabled; refine logprobs config handling.
• Error Handling:
  • Print full stack traces for caught exceptions in LiteLLM/OpenAI paths.
• Version:
  • Bump __version__ and project version to 0.10.0.dev12.

^{Written by Cursor Bugbot for commit d98842e. This will update automatically on new commits. Configure here.}

[kennysong]

bugbot run

[Copilot]

Pull Request Overview

This PR adds support for OpenAI's reasoning summary feature to the OpenAI evaluation clients, along with enhanced error logging using traceback for better debugging. The changes introduce new parameters to enable reasoning summaries and refactor the API dispatch logic to support both the Chat Completions API and the Responses API.

Key changes:

• Added reasoning summary parameters (use_reasoning_summary, reasoning_effort, reasoning_summary) to OpenAIEvalClient and AzureOpenAIEvalClient
• Introduced a new _dispatch method to handle routing between Chat Completions API and Responses API
• Enhanced exception handling with traceback.print_exception() calls in multiple places
• Updated response processing logic to extract and format reasoning summaries

Reviewed Changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

File	Description
src/langcheck/metrics/eval_clients/_openai.py	Added reasoning summary support with new parameters, refactored API dispatch logic, enhanced error logging with traceback
src/langcheck/metrics/eval_clients/_litellm.py	Added traceback printing for better error debugging

Comments suppressed due to low confidence (3)

src/langcheck/metrics/eval_clients/_openai.py:554

• Corrected spelling of 'Intialize' to 'Initialize'.

        Intialize the Azure OpenAI evaluation client.

src/langcheck/metrics/eval_clients/_openai.py:56

• The docstring is missing documentation for the newly added parameters: use_reasoning_summary, reasoning_effort, and reasoning_summary. These should be documented to explain their purpose and usage.

            openai_client (Optional): The OpenAI client to use.
            openai_args (Optional): dict of additional args to pass in to the
            `client.chat.completions.create` function.
            use_async: If True, the async client will be used. Defaults to
                False.
            system_prompt (Optional): The system prompt to use. If not provided,
                no system prompt will be used.
            extractor (Optional): The extractor to use. If not provided, the
                default extractor will be used.

src/langcheck/metrics/eval_clients/_openai.py:572

• The docstring is missing documentation for the newly added parameters: use_reasoning_summary, reasoning_effort, and reasoning_summary. These should be documented to explain their purpose and usage.

            text_model_name (Optional): The text model name you want to use with
                the Azure OpenAI API. The name is used as
                `{ "model": text_model_name }` parameter when calling the Azure
                OpenAI API for text models.
            embedding_model_name (Optional): The text model name you want to
                use with the Azure OpenAI API. The name is used as
                `{ "model": embedding_model_name }` parameter when calling the
                Azure OpenAI API for embedding models.
            azure_openai_client (Optional): The Azure OpenAI client to use.
            openai_args (Optional): dict of additional args to pass in to the
                `client.chat.completions.create` function.
            use_async (Optional): If True, the async client will be used.
            system_prompt (Optional): The system prompt to use. If not provided,
                no system prompt will be used.
            extractor (Optional): The extractor to use. If not provided, the
                default extractor will be used.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[kennysong]

Done reviewing!

[kennysong]

Ah nice, I was wondering if we set this properly to avoid logging prompts

[kennysong]

Let's also bump the version in this PR!

https://langcheck.readthedocs.io/en/latest/contributing.html#publishing

[kennysong]

LGTM after bumping the version!

[taniokay]

Thanks for your quick review!