[BUG] DeepSeek Reasoner rejects system messages through LiteLLM with invalid message format error

[RonanKMcGovern]

Describe the bug
When using LiteLLM to interact with DeepSeek Reasoner's API, the API rejects requests that include system messages with a 400 error. The error message indicates that "The last message of deepseek-reasoner must be a user message, or an assistant message with prefix mode on", suggesting that DeepSeek Reasoner has specific requirements about message ordering and format that aren't being handled correctly by the current LiteLLM integration.

Code to reproduce the error

from smolagents import CodeAgent, LiteLLMModel
import litellm

# Enable verbose mode for debugging
litellm.set_verbose = True

# Initialize the model
# model = LiteLLMModel("deepseek/deepseek-chat")
model = LiteLLMModel("deepseek/deepseek-reasoner")

# Create a basic agent
agent = CodeAgent(
    tools=[],
    model=model,
    add_base_tools=True,
    verbosity_level=2
)

# Run the agent with a simple task
result = agent.run("Get the weather in Dublin")
print(result) 

Error logs (if any)

uv run test-scripts/minimal_r1_agent.py
/Users/ronanmcgovern/TR/ADVANCED-inference/agentic-rag/.venv/lib/python3.12/site-packages/pydantic/_internal/_config.py:345: UserWarning: Valid config keys have changed in V2:
* 'fields' has been removed
  warnings.warn(message, UserWarning)
╭────────────────────────────────────────────────── New run ──────────────────────────────────────────────────╮
│                                                                                                             │
│ Get the weather in Dublin                                                                                   │
│                                                                                                             │
╰─ LiteLLMModel - deepseek/deepseek-reasoner ─────────────────────────────────────────────────────────────────╯
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Step 0 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
11:40:35 - LiteLLM:WARNING: utils.py:423 - `litellm.set_verbose` is deprecated. Please set `os.environ['LITELLM_LOG'] = 'DEBUG'` for debug logs.
SYNC kwargs[caching]: False; litellm.cache: None; kwargs.get('cache')['no-cache']: False
Final returned optional params: {'stop': ['<end_code>', 'Observation:'], 'max_tokens': 4096, 'extra_body': {}}
openai.py: Received openai error - error - Expecting value: line 1 column 1 (char 0), Received response - <APIResponse [200 OK] type=<class 'openai.types.chat.chat_completion.ChatCompletion'>>, Type of response - <class 'openai._legacy_response.LegacyAPIResponse'>
RAW RESPONSE:
error - Expecting value: line 1 column 1 (char 0), Received response - <APIResponse [200 OK] type=<class 'openai.types.chat.chat_completion.ChatCompletion'>>, Type of response - <class 'openai._legacy_response.LegacyAPIResponse'>



Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'.


Provider List: https://docs.litellm.ai/docs/providers

Error in generating model output:
litellm.APIError: APIError: DeepseekException - error - Expecting value: line 1 column 1 (char 0), Received 
response - <APIResponse [200 OK] type=<class 'openai.types.chat.chat_completion.ChatCompletion'>>, Type of 
response - <class 'openai._legacy_response.LegacyAPIResponse'>
[Step 0: Duration 60.54 seconds]
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Step 1 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
11:41:35 - LiteLLM:WARNING: utils.py:423 - `litellm.set_verbose` is deprecated. Please set `os.environ['LITELLM_LOG'] = 'DEBUG'` for debug logs.
SYNC kwargs[caching]: False; litellm.cache: None; kwargs.get('cache')['no-cache']: False
Final returned optional params: {'stop': ['<end_code>', 'Observation:'], 'max_tokens': 4096, 'extra_body': {}}
openai.py: Received openai error - Error code: 400 - {'error': {'message': 'The last message of deepseek-reasoner must be a user message, or an assistant message with prefix mode on (refer to https://api-docs.deepseek.com/guides/chat_prefix_completion).', 'type': 'invalid_request_error', 'param': None, 'code': 'invalid_request_error'}}
RAW RESPONSE:
Error code: 400 - {'error': {'message': 'The last message of deepseek-reasoner must be a user message, or an assistant message with prefix mode on (refer to https://api-docs.deepseek.com/guides/chat_prefix_completion).', 'type': 'invalid_request_error', 'param': None, 'code': 'invalid_request_error'}}



Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'.


Provider List: https://docs.litellm.ai/docs/providers

Error in generating model output:
litellm.BadRequestError: DeepseekException - Error code: 400 - {'error': {'message': 'The last message of 
deepseek-reasoner must be a user message, or an assistant message with prefix mode on (refer to 
https://api-docs.deepseek.com/guides/chat_prefix_completion).', 'type': 'invalid_request_error', 'param': None,
'code': 'invalid_request_error'}}
[Step 1: Duration 0.47 seconds]
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Step 2 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
11:41:36 - LiteLLM:WARNING: utils.py:423 - `litellm.set_verbose` is deprecated. Please set `os.environ['LITELLM_LOG'] = 'DEBUG'` for debug logs.
SYNC kwargs[caching]: False; litellm.cache: None; kwargs.get('cache')['no-cache']: False
Final returned optional params: {'stop': ['<end_code>', 'Observation:'], 'max_tokens': 4096, 'extra_body': {}}
openai.py: Received openai error - Error code: 400 - {'error': {'message': 'The last message of deepseek-reasoner must be a user message, or an assistant message with prefix mode on (refer to https://api-docs.deepseek.com/guides/chat_prefix_completion).', 'type': 'invalid_request_error', 'param': None, 'code': 'invalid_request_error'}}
RAW RESPONSE:
Error code: 400 - {'error': {'message': 'The last message of deepseek-reasoner must be a user message, or an assistant message with prefix mode on (refer to https://api-docs.deepseek.com/guides/chat_prefix_completion).', 'type': 'invalid_request_error', 'param': None, 'code': 'invalid_request_error'}}



Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'.


Provider List: https://docs.litellm.ai/docs/providers

Error in generating model output:
litellm.BadRequestError: DeepseekException - Error code: 400 - {'error': {'message': 'The last message of 
deepseek-reasoner must be a user message, or an assistant message with prefix mode on (refer to 
https://api-docs.deepseek.com/guides/chat_prefix_completion).', 'type': 'invalid_request_error', 'param': None,
'code': 'invalid_request_error'}}
[Step 2: Duration 0.38 seconds]
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Step 3 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
11:41:36 - LiteLLM:WARNING: utils.py:423 - `litellm.set_verbose` is deprecated. Please set `os.environ['LITELLM_LOG'] = 'DEBUG'` for debug logs.
SYNC kwargs[caching]: False; litellm.cache: None; kwargs.get('cache')['no-cache']: False
Final returned optional params: {'stop': ['<end_code>', 'Observation:'], 'max_tokens': 4096, 'extra_body': {}}
openai.py: Received openai error - Error code: 400 - {'error': {'message': 'The last message of deepseek-reasoner must be a user message, or an assistant message with prefix mode on (refer to https://api-docs.deepseek.com/guides/chat_prefix_completion).', 'type': 'invalid_request_error', 'param': None, 'code': 'invalid_request_error'}}
RAW RESPONSE:
Error code: 400 - {'error': {'message': 'The last message of deepseek-reasoner must be a user message, or an assistant message with prefix mode on (refer to https://api-docs.deepseek.com/guides/chat_prefix_completion).', 'type': 'invalid_request_error', 'param': None, 'code': 'invalid_request_error'}}



Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'.


Provider List: https://docs.litellm.ai/docs/providers

Error in generating model output:
litellm.BadRequestError: DeepseekException - Error code: 400 - {'error': {'message': 'The last message of 
deepseek-reasoner must be a user message, or an assistant message with prefix mode on (refer to 
https://api-docs.deepseek.com/guides/chat_prefix_completion).', 'type': 'invalid_request_error', 'param': None,
'code': 'invalid_request_error'}}
[Step 3: Duration 0.54 seconds]
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Step 4 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
11:41:37 - LiteLLM:WARNING: utils.py:423 - `litellm.set_verbose` is deprecated. Please set `os.environ['LITELLM_LOG'] = 'DEBUG'` for debug logs.
SYNC kwargs[caching]: False; litellm.cache: None; kwargs.get('cache')['no-cache']: False
Final returned optional params: {'stop': ['<end_code>', 'Observation:'], 'max_tokens': 4096, 'extra_body': {}}
openai.py: Received openai error - Error code: 400 - {'error': {'message': 'The last message of deepseek-reasoner must be a user message, or an assistant message with prefix mode on (refer to https://api-docs.deepseek.com/guides/chat_prefix_completion).', 'type': 'invalid_request_error', 'param': None, 'code': 'invalid_request_error'}}
RAW RESPONSE:
Error code: 400 - {'error': {'message': 'The last message of deepseek-reasoner must be a user message, or an assistant message with prefix mode on (refer to https://api-docs.deepseek.com/guides/chat_prefix_completion).', 'type': 'invalid_request_error', 'param': None, 'code': 'invalid_request_error'}}



Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'.


Provider List: https://docs.litellm.ai/docs/providers

Error in generating model output:
litellm.BadRequestError: DeepseekException - Error code: 400 - {'error': {'message': 'The last message of 
deepseek-reasoner must be a user message, or an assistant message with prefix mode on (refer to 
https://api-docs.deepseek.com/guides/chat_prefix_completion).', 'type': 'invalid_request_error', 'param': None,
'code': 'invalid_request_error'}}
[Step 4: Duration 0.39 seconds]
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Step 5 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
11:41:37 - LiteLLM:WARNING: utils.py:423 - `litellm.set_verbose` is deprecated. Please set `os.environ['LITELLM_LOG'] = 'DEBUG'` for debug logs.
SYNC kwargs[caching]: False; litellm.cache: None; kwargs.get('cache')['no-cache']: False
Final returned optional params: {'stop': ['<end_code>', 'Observation:'], 'max_tokens': 4096, 'extra_body': {}}
openai.py: Received openai error - Error code: 400 - {'error': {'message': 'The last message of deepseek-reasoner must be a user message, or an assistant message with prefix mode on (refer to https://api-docs.deepseek.com/guides/chat_prefix_completion).', 'type': 'invalid_request_error', 'param': None, 'code': 'invalid_request_error'}}
RAW RESPONSE:
Error code: 400 - {'error': {'message': 'The last message of deepseek-reasoner must be a user message, or an assistant message with prefix mode on (refer to https://api-docs.deepseek.com/guides/chat_prefix_completion).', 'type': 'invalid_request_error', 'param': None, 'code': 'invalid_request_error'}}



Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'.


Provider List: https://docs.litellm.ai/docs/providers

Error in generating model output:
litellm.BadRequestError: DeepseekException - Error code: 400 - {'error': {'message': 'The last message of 
deepseek-reasoner must be a user message, or an assistant message with prefix mode on (refer to 
https://api-docs.deepseek.com/guides/chat_prefix_completion).', 'type': 'invalid_request_error', 'param': None,
'code': 'invalid_request_error'}}
[Step 5: Duration 0.53 seconds]
Reached max steps.
11:41:37 - LiteLLM:WARNING: utils.py:423 - `litellm.set_verbose` is deprecated. Please set `os.environ['LITELLM_LOG'] = 'DEBUG'` for debug logs.
SYNC kwargs[caching]: False; litellm.cache: None; kwargs.get('cache')['no-cache']: False
Final returned optional params: {'max_tokens': 4096, 'extra_body': {}}
RAW RESPONSE:
{"id": "f92067fc-3f17-414a-85b1-e85820353bd3", "choices": [{"finish_reason": "stop", "index": 0, "logprobs": null, "message": {"content": "I don't have real-time weather data access, but here's how to get the current Dublin weather:\n\n1. **Search Engine**: Try \"Dublin weather\" on Google/Bing for instant forecasts\n2. **Weather Services**: Use dedicated platforms like:\n   - AccuWeather: [https://www.accuweather.com](https://www.accuweather.com)\n   - BBC Weather: [https://www.bbc.co.uk/weather](https://www.bbc.co.uk/weather)\n3. **Mobile Apps**: Check default weather apps or install reputable ones like Weather Underground\n\nThis error occurred because I'm an AI text model without live internet access. For future weather requests, I'll always remind you to check official sources, as real-time data requires specialized APIs I don't possess.", "refusal": null, "role": "assistant", "audio": null, "function_call": null, "tool_calls": null, "reasoning_content": "Okay, let me try to figure out how to help the user get the weather in Dublin. The user's previous attempts resulted in several errors related to the API they were using, specifically mentioning Deepseek's requirements about message formats. The main error seems to be that the last message in the conversation must be a user message or an assistant message with prefix mode enabled. \n\nFirst, I need to understand the problem. The agent was probably trying to use the Deepseek API but didn't structure the messages correctly. The errors indicate that the API expects the last message to be from the user, or if it's an assistant message, prefix mode should be on. Since the agent kept getting the same error, maybe they were sending an assistant message as the last one without the proper prefix.\n\nSo, to fix this, I should structure the API request according to Deepseek's guidelines. That means ensuring the last message is from the user. Alternatively, if the assistant needs to send a message, they should enable prefix mode. But since the task is to get the weather, which requires external data, the assistant might not have real-time access. Therefore, the correct approach would be to inform the user that real-time weather data isn't accessible and suggest checking a reliable source or using a weather service. \n\nWait, but maybe the initial agent tried to generate a weather report directly, leading to API errors because it's not a function they can perform. The repeated 400 errors suggest the message format was wrong, but after several retries, the max steps were reached. So, the solution is to acknowledge the inability to fetch real-time data and guide the user to appropriate resources. That way, the response complies with the API's message structure by ending with a user message or properly formatted assistant response."}}], "created": 1738237298, "model": "deepseek-reasoner", "object": "chat.completion", "service_tier": null, "system_fingerprint": "fp_7e73fd9a08", "usage": {"completion_tokens": 524, "prompt_tokens": 1482, "total_tokens": 2006, "completion_tokens_details": {"accepted_prediction_tokens": null, "audio_tokens": null, "reasoning_tokens": 359, "rejected_prediction_tokens": null}, "prompt_tokens_details": {"audio_tokens": null, "cached_tokens": 0}, "prompt_cache_hit_tokens": 0, "prompt_cache_miss_tokens": 1482}}


Final answer: I don't have real-time weather data access, but here's how to get the current Dublin weather:

1. **Search Engine**: Try "Dublin weather" on Google/Bing for instant forecasts
2. **Weather Services**: Use dedicated platforms like:
   - AccuWeather: [https://www.accuweather.com](https://www.accuweather.com)
   - BBC Weather: [https://www.bbc.co.uk/weather](https://www.bbc.co.uk/weather)
3. **Mobile Apps**: Check default weather apps or install reputable ones like Weather Underground

This error occurred because I'm an AI text model without live internet access. For future weather requests, 
I'll always remind you to check official sources, as real-time data requires specialized APIs I don't possess.
[Step 6: Duration 0.53 seconds| Input tokens: 1,482 | Output tokens: 524]
I don't have real-time weather data access, but here's how to get the current Dublin weather:

1. **Search Engine**: Try "Dublin weather" on Google/Bing for instant forecasts
2. **Weather Services**: Use dedicated platforms like:
   - AccuWeather: [https://www.accuweather.com](https://www.accuweather.com)
   - BBC Weather: [https://www.bbc.co.uk/weather](https://www.bbc.co.uk/weather)
3. **Mobile Apps**: Check default weather apps or install reputable ones like Weather Underground

This error occurred because I'm an AI text model without live internet access. For future weather requests, I'll always remind you to check official sources, as real-time data requires specialized APIs I don't possess.

Expected behavior
Should search for weather and respond based on search results.

Packages version:

smolagents==1.6.0

Additional context
Note that a simple LiteLLM-only test script like this works fine:

from litellm import completion
import os

# Try a simple completion with DeepSeek Reasoner
messages = [
    {"role": "system", "content": "You are a helpful AI assistant that gives very concise answers like a pirate."},
    {"role": "user", "content": "Say hello!"}
]

try:
    response = completion(
        model="deepseek/deepseek-reasoner",
        messages=messages
    )
    print(response)
except Exception as e:
    print(f"Error: {str(e)}") 

[elsolo5000-2]

if you using ollama, it is because litellm include format:json in the request, wich is bugging on ollama with this model actually. i made a custom model and it is warking

[RonanKMcGovern]

I'm using litellm directly.

…

On Sat, Feb 1, 2025 at 2:32 PM elsolo5000-2 ***@***.***> wrote: if you using ollama, it is because litellm include format:json in the request, wich is bugging on ollama with this model actually. i made a custom model and it is warking — Reply to this email directly, view it on GitHub <#430 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ASVG6CXS5UIRBKNOIW7PEEL2NTLHXAVCNFSM6AAAAABWFBV4D2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMMRYHE3TKNRWGY> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[elsolo5000-2]

I will compare litellm output with the deepseek doc to see. Last time i checked they re repo they where aving problems too. I think someone made a special model id prefix for reasoner models on the dev version

[elsolo5000-2]

Error code: 400 - {'error': {'message': 'The last message of deepseek-reasoner must be a user message, or an assistant message with prefix mode on (refer to https://api-docs.deepseek.com/guides/chat_prefix_completion).', 'type': 'invalid_request_error', 'param': None, 'code': 'invalid_request_error'}}

litellm seems to output the correct format, but perhaps we send a chat history with role:assistant as last message, wich is a beta feature of deepseek reasoner.

see
https://api-docs.deepseek.com/guides/chat_prefix_completion

i did not tried on deepseek api i just watched logs from wireshark on a personal endpoint