Chapter 5: Response Synthesis - Natural Language from Tool Results

[jrbrowning]

Natural Language in the Workshop

Chapter 5 implements the required pattern from the OpenAI specification: tool call request → execution → synthesis to natural language. This represents one complete round of interaction, not a loop.

(missed a chapter? You can find them here. The main branch will always contain the latest code, synchronized with the latest released chapter. You can review earlier chapters by selecting the branch matching the chapter title.)

Synthesis Pattern as a Building Block

The synthesis pattern serves as a fundamental building block. While you could use it to construct many different architectures, the pattern itself isn't an agent—it's the two-phase execution that the OpenAI specification defines.
For more background, see function calling in the OpenAI documentation.

Understanding "Synthesis"

In this tutorial, we deliberately use the term "synthesis" to describe this action. A tool call can end with only a tool result return, but synthesis represents the second stage where:

1. The tool result is returned to the model along with the tool definition
2. A tool_call_id syncs the two components
3. The model processes this second LLM call and returns natural language (instead of just the raw result)

By defining this term explicitly, we can establish different strategies from our frontend (covered in chapters 7-9) depending on the intended action: returning either a tool result (JSON format) or natural language representation of the tool result (text format).

Retrospective: Building on Chapter 4

In Chapter 4, we introduced the base tool calling pattern. We defined a function using the OpenAI-compatible specification and passed it along with our prompt and model parameters.

Our system prompt in the test case specified that the tool should explicitly call the get_weather function. Most models respect this query and return a result. However, now that we have our tool system prompt encapsulated within our tool definition, we don't need to state this twice—once in the system prompt in the test case, and again inside the tool definition.

This encapsulation enables our system prompt from the input perspective to focus exclusively on tone and presentation requirements. For this chapter, our tool call results will return our generic get_weather result as "Answer as a science fiction writer."

The test case includes an additional Pydantic dataclass strategy called "synthesis" (along with "tool_call") that routes requests to endpoints containing tool definitions. Separating this into a strategy that accompanies the request enables selection based on user intention.

    ToolchainTestCase(
        id="synthesis_completion_1",
        stage_a=ToolchainModelStage(
            system_prompt=(
                "Answer as a science fiction writer."
            ),
            user_prompt="What's going on in Washington, DC today?",
            model_container="traditional",
            stream=False,
            prompt_tool_spec=ToolchainPromptSpec(
                strategy="tool_call",
                max_tokens=1024,
                temperature=0.0,
            ),
            prompt_synthesis_spec=ToolchainPromptSpec( <-- HERE
                strategy="synthesis",
                max_tokens=1024,
                temperature=1.0,
            ),
            synthesis=True, <-- HERE, we set to True
        ),
        evaluation=False,
    ),

Test Your Tool Calling

Run your first multi-model comparison with: poe test. This executes the same configuration across all five models updated to run the synthesis strategy in your CLI.

The architecture diagrams below illustrate the 2-step pattern in action.
Tool call + natural language response = synthesis

Streaming Tool call + natural language response = synthesis

Course Prompts

For this chapter, try using these prompts:

Option 1: Copilot - Explain something in the codebase:
ex: @workspace Where are system_prompts defined in this code base?
ex: @workspace explain how these 2 system prompts get combined together. Show me the file.
ex: @workspace explain how the get_weather_tool works in the synthesis operation.
ex: @workspace explain how I can add more tools inside toolchain_completion

Option 2: Local Models - RAG Experiment (You can read more about the idea Course Goal)

(use # in ollama to include file in context)
ex: #(select the file) tool_registry.py explain how I can add more tools inside toolchain_completion?

---

Questions? → Questions board