Add built-in support for tool control parameters #347

nwumnn · 2025-08-15T01:36:05Z

What this does

Adds tool choice and parallel tool calls options to the with_tool and with_tools method.

Users can now specify:

choice: control how the model should use tools (auto, none, required, or specific tool name)
parallel: boolean parameter to allow/disallow parallel tool calls

chat.with_tools(MyTool, choice: :required, parallel: false)
chat.with_tool(SpecificTool, choice: :specific_tool, parallel: false)

Also updates the handle_tool_calls method. With :required or specific tool choices, the tool_choice is automatically reset to nil after tool execution to prevent infinite loops.

Type of change

Scope check

I read the Contributing Guide
This aligns with RubyLLM's focus on LLM communication
This isn't application-specific logic that belongs in user code
This benefits most users, not just my specific use case

Quality check

I ran overcommit --install and all hooks pass
I tested my changes thoroughly
I updated documentation if needed
I didn't modify auto-generated files manually (models.json, aliases.json)

API changes

Breaking change
New public methods/classes
Changed method signatures
No API changes

Related issues

Fixes #343

- Implement provider-specific tool choice handling - Add InvalidToolChoiceError for validation - Enhance tool execution flow to prevent infinite loops with non-auto choices

crmne · 2025-08-26T09:29:59Z

docs/_core_features/tools.md

+# Choice options
+chat.with_tool(Weather, choice: :auto)    # Model decides whether to call any provided tools or not (default)
+chat.with_tool(Weather, choice: :any)     # Model must use one of the provided tools
+chat.with_tool(Weather, choice: :none)    # No tools
+chat.with_tool(Weather, choice: :weather) # Force specific tool
+
+# Parallel tool calls
+chat.with_tools(Weather, Calculator, parallel: true)  # Model can output multiple tool calls at once (default)
+chat.with_tools(Weather, Calculator, parallel: false) # At most one tool call
+```


Both examples of with_tool and with_tools should contain both choice and parallel parameters otherwise people get the false sense that one parameter is for one call only.

crmne · 2025-08-26T09:39:32Z

lib/ruby_llm/chat.rb

-      halt_result || complete(&)
+      return halt_result if halt_result
+
+      should_continue_after_tools? ? complete(&) : response


The solution to this is not to halt the conversation, but to reset tool_choice to auto after tools have been called.

Halting the conversation is not normal behavior and it's for very specific use cases. More info here: https://community.openai.com/t/infinite-loop-with-tool-choice-required-or-type-function/755129

crmne · 2025-08-26T09:41:41Z

lib/ruby_llm/chat.rb


+    def update_tool_options(choice:, parallel:)
+      unless choice.nil?
+        valid_tool_choices = %i[auto none any] + tools.keys


as mentioned in the issue, I want the API to look like auto, none or required as required captures better what's going to happen.

crmne · 2025-08-26T09:44:07Z

lib/ruby_llm/chat.rb

    include Enumerable

-    attr_reader :model, :messages, :tools, :params, :headers, :schema
+    attr_reader :model, :messages, :tools, :tool_choice, :parallel_tool_calls, :params, :headers, :schema


Better to compress these two into one tool_prefs = {choice: ..., parallel: ...}

Should choice and parallel default to nil (like temperature) to use provider defaults, or should we set them explicitly to :auto and true?

Definitely set them to nil

crmne

Very good. These are finishing touches. It now needs tests.

crmne · 2025-08-28T12:54:00Z

docs/_core_features/tools.md

+Control when and how tools are called using `choice` and `parallel` options.
+
+**Parameter Values:**
+- **`choice`**: Controls tool choice behavior
+  - `:auto` Model decides whether to use any tools
+  - `:required` - Model must use one of the provided tools
+  - `:none` - Disable all tools
+  - `"tool_name"` - Force a specific tool (e.g., `:weather` for `Weather` tool)
+- **`parallel`**: Controls parallel tool calls
+  - `true` Allow multiple tool calls simultaneously
+  - `false` - One at a time
+
+If not provided, RubyLLM will use the provider's default behavior for tool choice and parallel tool calls.
+
+**Examples:**
+
+```ruby
+chat = RubyLLM.chat(model: 'gpt-4o')
+
+# Basic usage with defaults
+chat.with_tools(Weather, Calculator)  # uses provider defaults
+
+# Force tool usage, one at a time
+chat.with_tools(Weather, Calculator, choice: :required, parallel: false)
+
+# Force specific tool
+chat.with_tool(Weather, choice: :weather, parallel: true)


I'd prefer that to be code first, then parameter values. Also when specifying a tool in choice, it should accept the tool class too.

crmne · 2025-08-28T12:54:34Z

lib/ruby_llm/chat.rb

+      @tool_prefs = { choice: nil, parallel: nil }
      @messages = []
      @tools = {}


small thing, but make it after @tools

crmne · 2025-08-28T12:55:08Z

lib/ruby_llm/chat.rb

        params: @params,
        headers: @headers,
        schema: @schema,
+        tool_prefs: @tool_prefs,


pass it after tools

nwumnn · 2025-08-29T20:46:54Z

For the tests, I'm not sure what specific tests you'd like me to add. Any guidance would be great, thanks!

crmne · 2025-08-29T22:20:53Z

@nwumnn what's your experience with testing so far?

Here's how I would do it: add a couple tests that call the real APIs of the providers. Check chat_tools_spec.rb and add things there. One could be: add a tool then define tool choice none and check the tool didn't run. Another: add a tool that has no reason to run (weather while you ask when was the fall of Rome for example), but set it to required and check it has run. For parallel function calling set two tools that have to run because of the query and set parallel to false so the AI has to run one at a time.

nwumnn · 2025-08-30T16:55:59Z

Got it, thanks for the clear guidance! I'll add those tests to chat_tools_spec.rb

nwumnn · 2025-09-01T04:54:11Z

While writing tests, I found that some providers have limitations:

DeepSeek, Gemini, VertexAI, Bedrock: don't support disabling parallel tool calls
Ollama and GPUStack: supports neither tool choice nor parallel tool calls

Should I add methods like supports_tool_choice? and supports_tool_parallel_control? to the provider capability modules?

nwumnn added 3 commits August 14, 2025 16:55

feat: add tool choice and parallel tool calls control

af21430

- Implement provider-specific tool choice handling - Add InvalidToolChoiceError for validation - Enhance tool execution flow to prevent infinite loops with non-auto choices

docs: add Tool Choice Control section to tools.md

47923bc

Merge branch 'main' into tool-control

f1a6a4e

crmne requested changes Aug 26, 2025

View reviewed changes

nwumnn added 5 commits August 26, 2025 22:04

Reset tool_choice after forced tool use to prevent infinite loops

4a9efc4

Replace :any with :required

70831d1

Refactor tool options into unified tool_prefs hash

a932b78

Update docs

3a2b830

Merge branch 'main' into tool-control

54e9554

crmne requested changes Aug 28, 2025

View reviewed changes

nwumnn added 3 commits August 29, 2025 13:28

Pass tool_prefs after tools

fb06e7d

Update docs

fefd1a2

Merge branch 'main' into tool-control

8f34386

Merge branch 'main' into tool-control

9c6172b

nwumnn added 4 commits September 23, 2025 13:30

Merge branch 'main' into tool-control

d5d5389

Namespace build_tool_choice with Tools module

5359c79

Add tool control capability methods to providers

cef8c06

Add tests

9a3d682

nwumnn requested a review from crmne September 25, 2025 16:59

Uh oh!

Add built-in support for tool control parameters #347

Are you sure you want to change the base?

Add built-in support for tool control parameters #347

Uh oh!

Conversation

nwumnn commented Aug 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What this does

Type of change

Scope check

Quality check

API changes

Related issues

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

crmne left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

nwumnn commented Aug 29, 2025

Uh oh!

crmne commented Aug 29, 2025

Uh oh!

nwumnn commented Aug 30, 2025

Uh oh!

nwumnn commented Sep 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

nwumnn commented Aug 15, 2025 •

edited

Loading

nwumnn commented Sep 1, 2025 •

edited

Loading