fix(api): forward vendor fields on chat messages and completions - #833
fix(api): forward vendor fields on chat messages and completions#833paultranvan wants to merge 4 commits into
Conversation
|
Important Review skippedDraft detected. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
📝 WalkthroughWalkthroughChangesOpenAI schema passthrough
Estimated code review effort: 2 (Simple) | ~10 minutes Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@openrag/api/schemas/user/chat.py`:
- Around line 7-13: Update the chat router’s request logging to avoid
serializing full request.messages now that UserMessage allows extras via
model_config. In the logging path using truncate(str(request.messages)), log
only safe message metadata or explicitly redact extra fields, while preserving
passthrough of fields needed by downstream LLM processing.
- Around line 7-15: Update the message schema’s role declaration in the model
containing model_config to accept "tool" alongside the existing roles, allowing
tool-result messages with tool_call_id to validate and pass through. Add a
regression test covering a role="tool" message that includes tool_call_id.
In `@tests/unit/api/schemas/test_api_schema_imports.py`:
- Around line 117-137: Extend
test_chat_message_passes_through_extra_openai_fields to assert preservation of
tool_calls[0].function.name and function.arguments, plus tool_call_id and
function_call using the corresponding message fixtures. Keep the existing name
and tool-call ID assertions, and ensure nested OpenAI fields are verified after
model_dump(exclude_none=True).
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro Plus
Run ID: 939bd10a-b875-4e34-bfa5-f0515a6e396b
📒 Files selected for processing (2)
openrag/api/schemas/user/chat.pytests/unit/api/schemas/test_api_schema_imports.py
| def test_chat_message_passes_through_extra_openai_fields(): | ||
| """An OpenAI message is more than role/content: `name` disambiguates speakers | ||
| and `tool_calls`/`tool_call_id` carry function calling. Dropping them here | ||
| silently truncated the history sent to the LLM | ||
| """ | ||
| request = OpenAIChatCompletionRequest.model_validate( | ||
| { | ||
| "messages": [ | ||
| {"role": "user", "content": "hi", "name": "alice"}, | ||
| { | ||
| "role": "assistant", | ||
| "content": "", | ||
| "tool_calls": [{"id": "c1", "type": "function", "function": {"name": "f", "arguments": "{}"}}], | ||
| }, | ||
| ] | ||
| } | ||
| ) | ||
| messages = request.model_dump(exclude_none=True)["messages"] | ||
|
|
||
| assert messages[0]["name"] == "alice" | ||
| assert messages[1]["tool_calls"][0]["id"] == "c1" |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "Files matching target:"
git ls-files 'tests/unit/api/schemas/test_api_schema_imports.py' || true
if [ -f tests/unit/api/schemas/test_api_schema_imports.py ]; then
echo
echo "Outline/target section:"
wc -l tests/unit/api/schemas/test_api_schema_imports.py
sed -n '1,220p' tests/unit/api/schemas/test_api_schema_imports.py | cat -n
fiRepository: linagora/openrag
Length of output: 10310
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "Chat schema files:"
fd -a 'chat\.py$' openrag api tests | sed 's#^\./##' | sort
echo
for f in $(fd 'chat\.py$' openrag | sort); do
echo "--- $f ($(wc -l < "$f") lines) ---"
cat -n "$f"
done
echo
echo "References to OpenAIMessage / tool_call_id / function_call in api.schemas:"
rg -n "OpenAIMessage|tool_call_id|function_call|name|extra|model_config|ConfigDict" api/schemas -g '*.py'Repository: linagora/openrag
Length of output: 37832
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "Pydantic availability/version:"
python3 - <<'PY'
try:
import pydantic
import pydantic.version
print(pydantic.__version__, pydantic.version.version_info())
except Exception as e:
print("pydantic import failed:", type(e).__name__, str(e))
PY
echo
if python3 - <<'PY'
import pydantic
print(pydantic.__version__)
PY
then
python3 - <<'PY'
from pydantic import BaseModel, ConfigDict
import json
class Inner(BaseModel):
model_config = ConfigDict(extra="allow")
role: str
content: str
class Request(BaseModel):
model_config = ConfigDict(extra="allow")
messages: list[Inner]
data = {
"messages": [
{"role": "user", "content": "hi", "name": "alice"},
{
"role": "assistant",
"content": "",
"tool_calls": [
{
"id": "c1",
"type": "function",
"tool_call_id": "t1",
"function": {"name": "f", "arguments": "{}"}
}
],
"function_call": {"name": "g"}
},
{"role": "tool", "tool_call_id": "t1", "content": "response"},
]
}
request = Request.model_validate(data)
dump = request.model_dump(exclude_none=True)
print(json.dumps(dump, indent=2))
checks = {
"name": dump["messages"][0].get("name"),
"tool_call_id": dump["messages"][1]["tool_calls"][0].get("tool_call_id"),
"function_call": dump["messages"][1].get("function_call"),
"nested_name": dump["messages"][1]["tool_calls"][0]["function"].get("name"),
"nested_arguments": dump["messages"][1]["tool_calls"][0]["function"].get("arguments"),
"tool_role_call_id": dump["messages"][2].get("tool_call_id"),
}
print("checks:", checks)
PY
fiRepository: linagora/openrag
Length of output: 378
🌐 Web query:
OpenAI chat completion assistant message tool_calls tool_call_id function_call tool role messages parameter
💡 Result:
In the OpenAI Chat Completions API, the interaction between tool calls and assistant messages is managed through specific parameters and message roles [1][2]. The tool_calls parameter is an array within an assistant message (role: "assistant") that contains the tool calls the model has requested to execute [1][3]. Each object in this array typically includes an id (tool_call_id), which acts as a unique identifier for that specific call, and the details of the function or tool to be invoked (e.g., function name and arguments) [4][5][6]. When the model generates a tool_calls request, the developer must execute the corresponding functions and return the results to the model [5]. This is done by appending a new message with the role "tool" to the message history [5]. This "tool" message must include: 1. tool_call_id: Must match the id from the original tool_calls request [5][7]. 2. content: The output of the function execution [4][5]. The older function_call parameter is deprecated in favor of tool_calls and tools [1][3][6]. While function_call was once the primary way to handle function execution, tool_calls provides a more robust and flexible system, allowing for multiple tool calls in a single turn [1][6]. A complete message flow typically looks like this: 1. User sends a prompt. 2. Assistant replies with a message containing tool_calls (an array of calls with unique tool_call_ids) [1]. 3. Developer appends this assistant message to the history and then appends one "tool" message for each tool call, matching the tool_call_id [5]. 4. Developer calls the Chat Completions API again with the updated message history, allowing the model to process the tool results and formulate a final response [5]. If you send an assistant message containing tool_calls, the API requires you to provide corresponding "tool" messages to satisfy the tool_call_id requirements, or it will return an error [7].
Citations:
- 1: https://github.com/openai/openai-python/blob/f16fbbd2/src/openai/types/chat/chat_completion_assistant_message_param.py
- 2: https://github.com/openai/openai-python/blob/5e8f09c2/src/openai/types/chat/chat_completion_assistant_message_param.py
- 3: https://github.com/openai/openai-python/blob/d231d1fa783967c1d3a1db3ba1b52647fff148ac/src/openai/types/chat/chat_completion_message.py
- 4: https://developers.openai.com/api/docs/guides/function-calling
- 5: https://developers.openai.com/cookbook/examples/how_to_call_functions_with_chat_models/
- 6: https://community.openai.com/t/what-is-deference-between-function-call-and-tool-call/686481
- 7: https://community.openai.com/t/how-to-create-an-assistant-message-after-a-function-call-in-the-new-tools-api/578020
🌐 Web query:
site:openai.github.io/openai-python ChatCompletionMessageParam tool role
💡 Result:
ChatCompletionMessageParam can include a message with role: "tool" for tool results — i.e. the output returned after the assistant makes a tool call. In the OpenAI Python SDK internals, tool messages are paired using tool_call_id, and the model expects the tool-result message to reference the original tool call ID. (openai.github.io)
If you want, I can show the exact JSON shape for a tool message.
Citations:
🌐 Web query:
OpenAI Python chat completions tool role messages role tool tool_call_id required
💡 Result:
In the OpenAI Python Chat Completions API, when providing the result of a tool call back to the model, you must use a message with the role set to "tool" [1][2]. For this "tool" role message, the following fields are required: 1. role: Must be the literal string "tool" [1][3]. 2. tool_call_id: A string matching the id of the specific tool call you are responding to [1][2][4]. 3. content: The output of the tool execution (as a string) [1][2]. Additionally, while not strictly listed as a required parameter in some schema definitions compared to the three above, it is standard practice to include the name field (the name of the tool function that was called) in your message dictionary to ensure correct identification by the model [2][5][4]. Example structure for appending a tool result to your messages list: messages.append({ "role": "tool", "tool_call_id": tool_call.id, "name": tool_call.function.name, "content": json.dumps(function_result) }) It is important to note that a message with the role "tool" must directly follow the assistant's message that requested the tool call, and the tool_call_id must strictly correspond to the id provided in that assistant message [2][4]. Attempting to reuse an object type like ChatCompletionMessage (which is typically for assistant responses) for this "tool" role message will often result in a validation error, as the role "tool" is specifically required [5].
Citations:
- 1: https://github.com/openai/openai-python/blob/main/src/openai/types/chat/chat_completion_tool_message_param.py
- 2: https://developers.openai.com/cookbook/examples/how_to_call_functions_with_chat_models/
- 3: https://apis.io/schemas/openai-apis/openai-apis-chatmessage/
- 4: https://learn.microsoft.com/en-us/azure/foundry/openai/how-to/function-calling
- 5: It would be useful if ChatCompletionMessage could be used everywhere a chat message is needed. openai/openai-python#1358
Cover the full message passthrough contract.
This test only checks name and tool_calls[0]["id"], leaving nested fields such as tool_call_id, function_call, and nested function.name/function.arguments unguarded. Add assertions for those fields in this case or an equivalent tool-result message case.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@tests/unit/api/schemas/test_api_schema_imports.py` around lines 117 - 137,
Extend test_chat_message_passes_through_extra_openai_fields to assert
preservation of tool_calls[0].function.name and function.arguments, plus
tool_call_id and function_call using the corresponding message fixtures. Keep
the existing name and tool-call ID assertions, and ensure nested OpenAI fields
are verified after model_dump(exclude_none=True).
1 participant
Ported from a production hotfix: OpenRag was silently dropping OpenAI fields it
did not declare, so they never reached the downstream LLM.
OpenAIMessageAn OpenAI message is more than
role/content—name,tool_calls,function_call,tool_call_id. Pydantic's defaultextra="ignore"dropped allof them at parse time, before the router dumped the payload, so the history
forwarded to the model was silently truncated.
OpenAIChatCompletionRequestalready carriesextra="allow"with the comment"Accept and forward vendor-specific OpenAI params" — this applies the same policy
one level down, where the vendor fields actually live.
Concretely,
QueryService._sanitize_messagesalready branches onmsg.get("tool_calls")/msg.get("function_call")to leave a content-freeassistant turn alone. Those branches were unreachable: the schema stripped the
fields before the sanitizer ever saw them.
OpenAICompletionRequestSame passthrough on the legacy
/completionsendpoint, for consistency.extra="allow"only admits undeclared keys, so the deliberaten/best_ofbounds (resource-exhaustion guard) still validate — covered by a test.
Tests
Three unit tests in
tests/unit/api/schemas/test_api_schema_imports.py:name,tool_calls) survivemodel_dump_sanitize_messagespreserves atool_callsassistant turn end-to-end/completionsforwardssuffix/userwhile still rejectingn=9Full unit suite: no new failures (2204 → 2207 passing).
Summary by CodeRabbit
New Features
Bug Fixes