Skip to content

Commit d85133e

Browse files
committed
feat: update docs and add tool choice configuration section
1 parent 79a67aa commit d85133e

File tree

1 file changed

+58
-2
lines changed

1 file changed

+58
-2
lines changed

docs/source/basic_tutorials/using_guidance.md

Lines changed: 58 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -315,8 +315,6 @@ print(chat.choices[0].message.tool_calls)
315315

316316
TGI exposes an OpenAI-compatible API, which means you can use OpenAI's client libraries to interact with TGI's Messages API and Tool functions.
317317

318-
However there are some minor differences in the API, for example `tool_choice="auto"` will ALWAYS choose the tool for you. This is different from OpenAI's API where `tool_choice="auto"` will choose a tool if the model thinks it's necessary.
319-
320318
```python
321319
from openai import OpenAI
322320

@@ -362,3 +360,61 @@ print(called)
362360
# },
363361
# }
364362
```
363+
364+
### Tool Choice Configuration
365+
366+
When configuring how the model interacts with tools during a chat completion, there are several options for determining if or how a tool should be called. These options are controlled by the `tool_choice` parameter, which specifies the behavior of the model in relation to tool usage. The following modes are supported:
367+
368+
1. **`auto`**:
369+
370+
- The model decides whether to call a tool or generate a response message based on the user's input.
371+
- If tools are provided, this is the default mode.
372+
- Example usage:
373+
```python
374+
tool_choice="auto"
375+
```
376+
377+
2. **`none`**:
378+
379+
- The model will never call any tools and will only generate a response message.
380+
- If no tools are provided, this is the default mode.
381+
- Example usage:
382+
```python
383+
tool_choice="none"
384+
```
385+
386+
3. **`required`**:
387+
388+
- The model must call one or more tools and will not generate a response message on its own.
389+
- Example usage:
390+
```python
391+
tool_choice="required"
392+
```
393+
394+
4. **Specific Tool Call by Function Name**:
395+
- You can force the model to call a specific tool either by specifying the tool function directly or by using an object definition.
396+
- Two ways to do this:
397+
1. Provide the function name as a string:
398+
```python
399+
tool_choice="get_current_weather"
400+
```
401+
2. Use the function object format:
402+
```python
403+
tool_choice={
404+
"type": "function",
405+
"function": {
406+
"name": "get_current_weather"
407+
}
408+
}
409+
```
410+
411+
These options allow flexibility when integrating tools with the chat completions endpoint. You can configure the model to either rely on tools automatically or force it to follow a predefined behavior, based on the needs of the task at hand.
412+
413+
---
414+
415+
| **Tool Choice Option** | **Description** | **When to Use** |
416+
| ------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------- |
417+
| `auto` | The model decides whether to call a tool or generate a message. This is the default if tools are provided. | Use when you want the model to decide when a tool is necessary. |
418+
| `none` | The model generates a message without calling any tools. This is the default if no tools are provided. | Use when you do not want the model to call any tools. |
419+
| `required` | The model must call one or more tools and will not generate a message on its own. | Use when a tool call is mandatory, and you do not want a regular message generated. |
420+
| Specific Tool Call (`name` or object) | Force the model to call a specific tool either by specifying its name (`tool_choice="get_current_weather"`) or using an object. | Use when you want to restrict the model to calling a particular tool for the response. |

0 commit comments

Comments
 (0)