[Bug]: deepseek-reasoner and deepseek-chat Output issues

[RonanKMcGovern]

What happened?

To reproduce:

from smolagents import CodeAgent, LiteLLMModel
import litellm

# Enable verbose mode for debugging
litellm.set_verbose = True

# Initialize the model
model = LiteLLMModel("deepseek/deepseek-reasoner")

# Create a basic agent
agent = CodeAgent(
    model=model,
    add_base_tools=False,
    verbosity_level=2
)

# Run the agent with a simple task
result = agent.run("Write a simple hello world function in Python.")
print(result)

When running a simple smolagent, with deepseek-reasoner, I get a correct zeroth response but subsequent first and second responses give this:

╭────────────────────────────────────────────────────────── New run ──────────────────────────────────────────────────────────╮
│                                                                                                                             │
│ Write a simple hello world function in Python.                                                                              │
│                                                                                                                             │
╰─ LiteLLMModel - deepseek/deepseek-reasoner ─────────────────────────────────────────────────────────────────────────────────╯
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Step 0 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
17:20:51 - LiteLLM:WARNING: utils.py:423 - `litellm.set_verbose` is deprecated. Please set `os.environ['LITELLM_LOG'] = 'DEBUG'` for debug logs.
SYNC kwargs[caching]: False; litellm.cache: None; kwargs.get('cache')['no-cache']: False
Final returned optional params: {'stop': ['<end_code>', 'Observation:'], 'max_tokens': 4096, 'extra_body': {}}
openai.py: Received openai error - error - Expecting value: line 1 column 1 (char 0), Received response - <APIResponse [200 OK] type=<class 'openai.types.chat.chat_completion.ChatCompletion'>>, Type of response - <class 'openai._legacy_response.LegacyAPIResponse'>
RAW RESPONSE:
error - Expecting value: line 1 column 1 (char 0), Received response - <APIResponse [200 OK] type=<class 'openai.types.chat.chat_completion.ChatCompletion'>>, Type of response - <class 'openai._legacy_response.LegacyAPIResponse'>



Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'.


Provider List: https://docs.litellm.ai/docs/providers

Error in generating model output:
litellm.APIError: APIError: DeepseekException - error - Expecting value: line 1 column 1 (char 0), Received response - 
<APIResponse [200 OK] type=<class 'openai.types.chat.chat_completion.ChatCompletion'>>, Type of response - <class 
'openai._legacy_response.LegacyAPIResponse'>
[Step 0: Duration 61.16 seconds]
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Step 1 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
17:21:52 - LiteLLM:WARNING: utils.py:423 - `litellm.set_verbose` is deprecated. Please set `os.environ['LITELLM_LOG'] = 'DEBUG'` for debug logs.
SYNC kwargs[caching]: False; litellm.cache: None; kwargs.get('cache')['no-cache']: False
Final returned optional params: {'stop': ['<end_code>', 'Observation:'], 'max_tokens': 4096, 'extra_body': {}}
openai.py: Received openai error - Error code: 400 - {'error': {'message': 'The last message of deepseek-reasoner must be a user message, or an assistant message with prefix mode on (refer to https://api-docs.deepseek.com/guides/chat_prefix_completion).', 'type': 'invalid_request_error', 'param': None, 'code': 'invalid_request_error'}}
RAW RESPONSE:
Error code: 400 - {'error': {'message': 'The last message of deepseek-reasoner must be a user message, or an assistant message with prefix mode on (refer to https://api-docs.deepseek.com/guides/chat_prefix_completion).', 'type': 'invalid_request_error', 'param': None, 'code': 'invalid_request_error'}}



Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'.


Provider List: https://docs.litellm.ai/docs/providers

Error in generating model output:
litellm.BadRequestError: DeepseekException - Error code: 400 - {'error': {'message': 'The last message of deepseek-reasoner 
must be a user message, or an assistant message with prefix mode on (refer to 
https://api-docs.deepseek.com/guides/chat_prefix_completion).', 'type': 'invalid_request_error', 'param': None, 'code': 
'invalid_request_error'}}
[Step 1: Duration 0.52 seconds]
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Step 2 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
17:21:52 - LiteLLM:WARNING: utils.py:423 - `litellm.set_verbose` is deprecated. Please set `os.environ['LITELLM_LOG'] = 'DEBUG'` for debug logs.
SYNC kwargs[caching]: False; litellm.cache: None; kwargs.get('cache')['no-cache']: False
Final returned optional params: {'stop': ['<end_code>', 'Observation:'], 'max_tokens': 4096, 'extra_body': {}}
openai.py: Received openai error - Error code: 400 - {'error': {'message': 'The last message of deepseek-reasoner must be a user message, or an assistant message with prefix mode on (refer to https://api-docs.deepseek.com/guides/chat_prefix_completion).', 'type': 'invalid_request_error', 'param': None, 'code': 'invalid_request_error'}}
RAW RESPONSE:
Error code: 400 - {'error': {'message': 'The last message of deepseek-reasoner must be a user message, or an assistant message with prefix mode on (refer to https://api-docs.deepseek.com/guides/chat_prefix_completion).', 'type': 'invalid_request_error', 'param': None, 'code': 'invalid_request_error'}}



Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'.


Provider List: https://docs.litellm.ai/docs/providers

Error in generating model output:
litellm.BadRequestError: DeepseekException - Error code: 400 - {'error': {'message': 'The last message of deepseek-reasoner 
must be a user message, or an assistant message with prefix mode on (refer to 
https://api-docs.deepseek.com/guides/chat_prefix_completion).', 'type': 'invalid_request_error', 'param': None, 'code': 
'invalid_request_error'}}
[Step 2: Duration 0.41 seconds]
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Step 3 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
17:21:53 - LiteLLM:WARNING: utils.py:423 - `litellm.set_verbose` is deprecated. Please set `os.environ['LITELLM_LOG'] = 'DEBUG'` for debug logs.
SYNC kwargs[caching]: False; litellm.cache: None; kwargs.get('cache')['no-cache']: False
Final returned optional params: {'stop': ['<end_code>', 'Observation:'], 'max_tokens': 4096, 'extra_body': {}}
openai.py: Received openai error - Error code: 400 - {'error': {'message': 'The last message of deepseek-reasoner must be a user message, or an assistant message with prefix mode on (refer to https://api-docs.deepseek.com/guides/chat_prefix_completion).', 'type': 'invalid_request_error', 'param': None, 'code': 'invalid_request_error'}}
RAW RESPONSE:
Error code: 400 - {'error': {'message': 'The last message of deepseek-reasoner must be a user message, or an assistant message with prefix mode on (refer to https://api-docs.deepseek.com/guides/chat_prefix_completion).', 'type': 'invalid_request_error', 'param': None, 'code': 'invalid_request_error'}}



Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'.


Provider List: https://docs.litellm.ai/docs/providers

Error in generating model output:
litellm.BadRequestError: DeepseekException - Error code: 400 - {'error': {'message': 'The last message of deepseek-reasoner 
must be a user message, or an assistant message with prefix mode on (refer to 
https://api-docs.deepseek.com/guides/chat_prefix_completion).', 'type': 'invalid_request_error', 'param': None, 'code': 
'invalid_request_error'}}
[Step 3: Duration 0.36 seconds]
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Step 4 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
17:21:53 - LiteLLM:WARNING: utils.py:423 - `litellm.set_verbose` is deprecated. Please set `os.environ['LITELLM_LOG'] = 'DEBUG'` for debug logs.
SYNC kwargs[caching]: False; litellm.cache: None; kwargs.get('cache')['no-cache']: False
Final returned optional params: {'stop': ['<end_code>', 'Observation:'], 'max_tokens': 4096, 'extra_body': {}}
openai.py: Received openai error - Error code: 400 - {'error': {'message': 'The last message of deepseek-reasoner must be a user message, or an assistant message with prefix mode on (refer to https://api-docs.deepseek.com/guides/chat_prefix_completion).', 'type': 'invalid_request_error', 'param': None, 'code': 'invalid_request_error'}}
RAW RESPONSE:
Error code: 400 - {'error': {'message': 'The last message of deepseek-reasoner must be a user message, or an assistant message with prefix mode on (refer to https://api-docs.deepseek.com/guides/chat_prefix_completion).', 'type': 'invalid_request_error', 'param': None, 'code': 'invalid_request_error'}}



Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'.


Provider List: https://docs.litellm.ai/docs/providers

Error in generating model output:
litellm.BadRequestError: DeepseekException - Error code: 400 - {'error': {'message': 'The last message of deepseek-reasoner 
must be a user message, or an assistant message with prefix mode on (refer to 
https://api-docs.deepseek.com/guides/chat_prefix_completion).', 'type': 'invalid_request_error', 'param': None, 'code': 
'invalid_request_error'}}
[Step 4: Duration 0.49 seconds]
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Step 5 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
17:21:54 - LiteLLM:WARNING: utils.py:423 - `litellm.set_verbose` is deprecated. Please set `os.environ['LITELLM_LOG'] = 'DEBUG'` for debug logs.
SYNC kwargs[caching]: False; litellm.cache: None; kwargs.get('cache')['no-cache']: False
Final returned optional params: {'stop': ['<end_code>', 'Observation:'], 'max_tokens': 4096, 'extra_body': {}}
openai.py: Received openai error - Error code: 400 - {'error': {'message': 'The last message of deepseek-reasoner must be a user message, or an assistant message with prefix mode on (refer to https://api-docs.deepseek.com/guides/chat_prefix_completion).', 'type': 'invalid_request_error', 'param': None, 'code': 'invalid_request_error'}}
RAW RESPONSE:
Error code: 400 - {'error': {'message': 'The last message of deepseek-reasoner must be a user message, or an assistant message with prefix mode on (refer to https://api-docs.deepseek.com/guides/chat_prefix_completion).', 'type': 'invalid_request_error', 'param': None, 'code': 'invalid_request_error'}}



Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'.


Provider List: https://docs.litellm.ai/docs/providers

Error in generating model output:
litellm.BadRequestError: DeepseekException - Error code: 400 - {'error': {'message': 'The last message of deepseek-reasoner 
must be a user message, or an assistant message with prefix mode on (refer to 
https://api-docs.deepseek.com/guides/chat_prefix_completion).', 'type': 'invalid_request_error', 'param': None, 'code': 
'invalid_request_error'}}
[Step 5: Duration 0.51 seconds]
Reached max steps.
17:21:54 - LiteLLM:WARNING: utils.py:423 - `litellm.set_verbose` is deprecated. Please set `os.environ['LITELLM_LOG'] = 'DEBUG'` for debug logs.
SYNC kwargs[caching]: False; litellm.cache: None; kwargs.get('cache')['no-cache']: False
Final returned optional params: {'max_tokens': 4096, 'extra_body': {}}

With Deepseek-chat, even the first call fails:

╭────────────────────────────────────────────────────────── New run ──────────────────────────────────────────────────────────╮
│                                                                                                                             │
│ Write a simple hello world function in Python.                                                                              │
│                                                                                                                             │
╰─ LiteLLMModel - deepseek/deepseek-chat ─────────────────────────────────────────────────────────────────────────────────────╯
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Step 0 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
17:09:00 - LiteLLM:WARNING: utils.py:423 - `litellm.set_verbose` is deprecated. Please set `os.environ['LITELLM_LOG'] = 'DEBUG'` for debug logs.
SYNC kwargs[caching]: False; litellm.cache: None; kwargs.get('cache')['no-cache']: False
Final returned optional params: {'stop': ['<end_code>', 'Observation:'], 'max_tokens': 4096, 'extra_body': {}}
openai.py: Received openai error - error - Expecting value: line 1 column 1 (char 0), Received response - <APIResponse [200 OK] type=<class 'openai.types.chat.chat_completion.ChatCompletion'>>, Type of response - <class 'openai._legacy_response.LegacyAPIResponse'>
RAW RESPONSE:
error - Expecting value: line 1 column 1 (char 0), Received response - <APIResponse [200 OK] type=<class 'openai.types.chat.chat_completion.ChatCompletion'>>, Type of response - <class 'openai._legacy_response.LegacyAPIResponse'>



Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'.


Provider List: https://docs.litellm.ai/docs/providers

Error in generating model output:
litellm.APIError: APIError: DeepseekException - error - Expecting value: line 1 column 1 (char 0), Received response - 
<APIResponse [200 OK] type=<class 'openai.types.chat.chat_completion.ChatCompletion'>>, Type of response - <class 
'openai._legacy_response.LegacyAPIResponse'>
[Step 0: Duration 61.20 seconds]
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Step 1 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
17:10:02 - LiteLLM:WARNING: utils.py:423 - `litellm.set_verbose` is deprecated. Please set `os.environ['LITELLM_LOG'] = 'DEBUG'` for debug logs.
SYNC kwargs[caching]: False; litellm.cache: None; kwargs.get('cache')['no-cache']: False
Final returned optional params: {'stop': ['<end_code>', 'Observation:'], 'max_tokens': 4096, 'extra_body': {}}
RAW RESPONSE:
{"id": "1322d3a0-e0d6-4f0c-a8be-e092a68c9f5d", "choices": [{"finish_reason": "stop", "index": 0, "logprobs": null, "message": {"content": "Thought: I need to write a simple Python function that prints \"Hello, World!\". This is a basic task that doesn't require any external tools or libraries.\n\nCode:\n```py\ndef hello_world():\n    print(\"Hello, World!\")\n\nhello_world()\n```", "refusal": null, "role": "assistant", "audio": null, "function_call": null, "tool_calls": null}}], "created": 1738170602, "model": "deepseek-chat", "object": "chat.completion", "service_tier": null, "system_fingerprint": "fp_3a5770e1b4", "usage": {"completion_tokens": 60, "prompt_tokens": 2101, "total_tokens": 2161, "completion_tokens_details": null, "prompt_tokens_details": {"audio_tokens": null, "cached_tokens": 1472}, "prompt_cache_hit_tokens": 1472, "prompt_cache_miss_tokens": 629}}


Output message of the LLM: ────────────────────────────────────────────────────────────────────────────────────────────────────
Thought: I need to write a simple Python function that prints "Hello, World!". This is a basic task that doesn't require any   
external tools or libraries.                                                                                                   
                                                                                                                               
Code:                                                                                                                          
```py                                                                                                                          
def hello_world():                                                                                                             
    print("Hello, World!")                                                                                                     
                                                                                                                               
hello_world()                                                                                                                  

─ Executing this code: ──────────────────────────────────────────────────────────────────────────────────────────────────────
def hello_world():
print("Hello, World!")

hello_world()
─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Execution logs:
Hello, World!

Out: None
[Step 1: Duration 3.72 seconds| Input tokens: 2,101 | Output tokens: 60]
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Step 2 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
17:10:05 - LiteLLM:WARNING: utils.py:423 - litellm.set_verbose is deprecated. Please set os.environ['LITELLM_LOG'] = 'DEBUG' for debug logs.
SYNC kwargs[caching]: False; litellm.cache: None; kwargs.get('cache')['no-cache']: False
Final returned optional params: {'stop': ['<end_code>', 'Observation:'], 'max_tokens': 4096, 'extra_body': {}}
openai.py: Received openai error - error - Expecting value: line 1 column 1 (char 0), Received response - <APIResponse [200 OK] type=<class 'openai.types.chat.chat_completion.ChatCompletion'>>, Type of response - <class 'openai._legacy_response.LegacyAPIResponse'>
RAW RESPONSE:
error - Expecting value: line 1 column 1 (char 0), Received response - <APIResponse [200 OK] type=<class 'openai.types.chat.chat_completion.ChatCompletion'>>, Type of response - <class 'openai._legacy_response.LegacyAPIResponse'>

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'.

Provider List: https://docs.litellm.ai/docs/providers

Error in generating model output:
litellm.APIError: APIError: DeepseekException - error - Expecting value: line 1 column 1 (char 0), Received response -
<APIResponse [200 OK] type=<class 'openai.types.chat.chat_completion.ChatCompletion'>>, Type of response - <class
'openai._legacy_response.LegacyAPIResponse'>
[Step 2: Duration 60.89 seconds| Input tokens: 4,202 | Output tokens: 120]
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Step 3 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
17:11:06 - LiteLLM:WARNING: utils.py:423 - litellm.set_verbose is deprecated. Please set os.environ['LITELLM_LOG'] = 'DEBUG' for debug logs.
SYNC kwargs[caching]: False; litellm.cache: None; kwargs.get('cache')['no-cache']: False
Final returned optional params: {'stop': ['<end_code>', 'Observation:'], 'max_tokens': 4096, 'extra_body': {}}

### Relevant log output

```shell

Are you a ML Ops Team?

No

What LiteLLM version are you on ?

v1.59.9

Twitter / LinkedIn details

No response

[RonanKMcGovern]

Apologies, I dug further and this is likely a smolagents issue. New issue there: huggingface/smolagents#430