[Ollama] How to get structured output from local DeepSeek-r1 ?

[scriptdealer]

class CharactersList(BaseModel):
    characters: Annotated[List[str], None, "List of characters"]

# model_name = "deepseek-r1:14b"
model_name = "qwen2.5:14b"
llm_json_mode = ChatOllama(model=model_name, top_p=0.8, temperature=0.5, format=CharactersList.model_json_schema())
prompt = f"List all characters in {state['book']}"
parsed_response = CharactersList.model_validate_json(response.content)

While this code works with Qwen2.5 and Llama3.2, DeepSeek-r1 gives me an empty output.
This is with
ollama 0.4.7
langchain-ollama 0.2.2

So, is this a bug that should be reported, or there is a way to work this around?

[dosubot[bot]]

Hello, @scriptdealer! I'm here to help you with any bugs, questions, or contributions you might have. Let's tackle this issue together!

I found a similar unsolved discussion regarding issues with function calling with local Ollama, which might be related to your problem. You can check it out here ^[1].

To address your issue with the DeepSeek-r1 model, it seems that the with_structured_output method is recommended for handling structured output. This method allows you to define a schema using a Pydantic model, which can help in structuring the output. However, JSON mode is not supported for DeepSeek models, which might be causing the empty output ^[2].

Ensure that you are not using JSON mode with DeepSeek-r1 and check your model deployment settings. If the problem persists, you might need to consult additional documentation or support resources. Additionally, verify that your API key is set up correctly and that the langchain-deepseek-official package is installed as per the setup instructions ^[2].

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Other}

[scriptdealer]

ollama._types.ResponseError: registry.ollama.ai/library/deepseek-r1:14b does not support tools (status code: 400)

this is what I got for with_structured_output method

[zakarehman270]

ollama deepseek_r1 is very slow

[adocherty]

Try using .with_structured_output with method="json_schema".

For example:

from langchain_ollama import ChatOllama
from pydantic import BaseModel, Field
from langchain_core.output_parsers import JsonOutputParser, PydanticOutputParser

llm_model = ChatOllama(model="deepseek-r1", temperature=0)

class Joke(BaseModel):
    """Joke to tell user."""

    setup: str = Field(description="The setup of the joke")
    punchline: str = Field(description="The punchline to the joke")
    rating: int = Field(description="How funny the joke is, from 1 to 10")

structured_llm = llm_model.with_structured_output(Joke, method="json_schema")
structured_llm.invoke("Tell me a joke about wombats")

This works for me; however, DeepseekR1 only supports the new json_schema mode that was released in Dec 2024 (https://ollama.com/blog/structured-outputs), so ensure you have an up to date Ollama, as well as up-to-date versions of all langchain packages especially langchain-ollama>=0.2.3.

Note that I've found that the reasoning models not that great for this as they try and reason in the structure you've provided. Some thoughts (I haven't tested these):

1. put a structured field for the chain of thought in the structure
2. run the reasoning model first with free output, then ask a second LLM to structure it

[gingemonster]

I can get a structured response from R1 using the json_schema mode however the answer is just the thinking from the model not the final answer, anyone else getting this?

[adocherty]

I had the same issue too. I found that adding a heirarchical structure that contains a "think" property then the structure you want to work for me, for eample:

class Joke(BaseModel):
    """Joke to tell user."""

    setup: str = Field(description="The setup of the joke")
    punchline: str = Field(description="The punchline to the joke")
    rating: int = Field(description="How funny the joke is, from 1 to 10")


class JokeReasoning(BaseModel):
    """Think through and tell a good joke to tell"""

    think: str = Field(
        description="Think though how to tell a good joke about the subject"
    )
    joke: Joke = Field(description="The final Joke")

llm_model = ChatOllama(model="deepseek-r1", temperature=0)
structured_llm = llm_model.with_structured_output(JokeReasoning, method="json_schema")
output = structured_llm.invoke("Tell me a joke about kangaroos")
print(output.joke)

Let me know if this works for you, I'd be interested to know.

[gingemonster]

Thanks yes you were right "think" seemed to be what it needed to be called although I managed to combine into a single class sucessfully:

class StructuredReasoningResponse(BaseModel):
think: str = Field(
description="Think though how to answer the question"
)
answer: str = Field(description="The answer as text")
reasoning: str = Field(description="The reasoning for the answer")
# below ensures the answer returned is always a string before conversation as numberic answers were causing validation errors
@field_validator("answer", mode="before")
def ensure_string(cls, v):
return str(v)

[adocherty]

Great, glad it worked for you! Good info that it can be combined into a single class. I struggled to get that to work but I didn't spend long trying it; now I think about it perhaps I didn't call it "think" 🤔

[ksanthoshNT]

@gingemonster @adocherty

setting think in response format is not equal to thinking ( aka reasoning of the model) .

its just the json with think.

expected behaviour is : 1. think for sometime in normal text 2. output json tokens so that we can parse.

someone made this using grammer + llamacpp :
https://colab.research.google.com/drive/1s7hyNWuG-wMcbvqFANjkLUi_-gpcQQ2E

this is the actual way.

we have to wait for ollama to release this real reasoning happend and then json ouput, but not json output containing thinking attribute.

[nnWhisperer]

This would be a popular use-case, considering that reasoning models and structured outputs are good and popular in their own ways. I'm surprised why langchain still haven't provided a how-to documentation for this :/

[adocherty]

@ksanthoshNT perhaps I should have been clearer that this is an obvious hack. You're right that this hack to include a JSON think tag will be different to allow the LLM to think between tags and and separately have a structured output.

However, it's interesting that the LLM model's I've tried do appear to work having a JSON structured output with think as a JSON tag. These models have considerable powers of generatization, so maybe they are able to infer the change in format and put something that appears to be thinking in JSON. The question is how does this compare to performance when using this approach versus the trained XML tags? This is a research provlem.

Until the functionality you mention is available, another approach with thinking models is to perform a two step process:

1. Ask the question of the thinking model with no structural contraints;
2. Feed the results to a non-thinking LLM to extract the answer in a given structure.