You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 3dc75bd
Browse filesBrowse the repository at this point in the historyBrowse files
Load MCP tools on demand; cap derived summary trigger at 100k
Every model call re-sent every bound tool definition. All four bots carry Linear's 76 MCP tools
(~30k tokens) plus Serena and filesystem (~11k more), on every call, while Linear was called 19
times in ~3,200 tool calls over a week.
- MCP tools stay bound but are left out of model requests until the bot loads them with the new
load_tools tool (exact names or keyword query, max 15 per call); the system prompt lists them by
name. The loaded set lives in agent state for the rest of the run; loaded tools go last in the
tool list so a load keeps the cached prefix. Calling an unloaded tool is refused but loads it.
- defer_mcp_tools setting (default on) turns it off.
- MAX_DERIVED_SUMMARY_TRIGGER 200k -> 100k: on glm-5.3-flash's 1M window nothing was cleared
until ~133k of messages and calls passed 170k tokens.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Copy file name to clipboardExpand all lines: README.md
+7Lines changed: 7 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -466,6 +466,12 @@ Tools appear in the registry as `server__tool` (`Linear__create_issue`) with sou
466
466
In the bot editor, MCP tools are grouped by server: grant a whole server with one checkbox, or expand
467
467
it and pick individual tools. `approval_tools` and `TOOL_OUTPUT_CAP` apply to them like any other tool.
468
468
469
+
MCP tools load on demand (`defer_mcp_tools`, on by default). A server can bring dozens of tools with
470
+
long schemas (Linear's 76 come to ~30k tokens), and every model call re-sends every tool definition it
471
+
is offered. So a bot's system prompt lists its MCP tools by name only, and the bot calls `load_tools`
472
+
(by exact name, or by keyword `query`) to add the ones a task needs; they stay loaded for the rest of the
473
+
run. A call to a tool that was never loaded does not run, but loads it so the retry has its schema.
474
+
469
475
Whatever you authorize is shared by every bot given that tool: a bot calling Linear acts as you.
470
476
Use `approval_tools` for anything that writes.
471
477
@@ -634,6 +640,7 @@ name, if set, is the default the page shows and the value a reset returns to.
634
640
| Embeddings |`embedding_model`, `embedding_dims`|`provider:model` for semantic memory search (`openrouter:openai/text-embedding-3-small`/1536 using the OpenRouter key, `openai:text-embedding-3-small`/1536, `ollama:nomic-embed-text`/768). Empty turns semantic search off. Applies immediately: the memory store is reopened. |
635
641
| Run limits |`max_model_calls_per_run` (60), `max_repeated_tool_failures` (3), `max_bot_hops` (20) | Model turns per run before the agent stops with a notice (a bot can lower it in `model_settings.max_model_calls`; the seeded Chief of Staff uses 6); how many times one tool call may fail with the same arguments and error before the run stops with that error as the reply (0 = off); bot-to-bot mention chain limit per thread. `model_settings` also accepts `reasoning_effort` (set from the Effort select for models that declare levels), `temperature`, `max_tokens`. |
636
642
| Context |`tool_output_cap` (8000), `shell_output_cap` (4000) | Longest single tool result the model sees; shorter cap for `run_shell` so dumping files through the shell loses to `read_file` ranges. |
643
+
| Context |`defer_mcp_tools` (true) | Leave MCP tool definitions out of model calls until the bot loads them with `load_tools`; the system prompt lists their names. |
637
644
| Context |`context_trigger_tokens` (40000), `context_clear_at_least` (10000) | Once a run's messages pass the trigger, tool results from turns before the last two, and their call arguments, become a placeholder, oldest first; each clearing reclaims at least the second value so clearings are rare and the prompt cache stays warm. What the last two model turns fetched is never cleared. |
638
645
| Context |`summary_trigger_tokens` (60000), `summary_keep_messages` (24) | Older history folds into one structured summary by the bot's own model once the run's messages pass the trigger. Measured on the messages alone: the system prompt and tool schemas are not counted. |
639
646
| Context |`history_token_budget` (24000), `history_max_messages` (80) | Thread history included in a run's prompt. |
Copy file name to clipboardExpand all lines: backend/openbot/runtime/app_settings.py
+2Lines changed: 2 additions & 0 deletions
Original file line number
Diff line number
Diff line change
@@ -78,6 +78,8 @@ class Tunable:
78
78
"Longest single tool result the model sees; longer results are shortened and say so.", 500),
79
79
"shell_output_cap": Tunable("Context", "Shell output cap (chars)",
80
80
"Tighter cap for run_shell, so dumping a file through the shell loses to read_file with a line range.", 500),
81
+
"defer_mcp_tools": Tunable("Context", "Load MCP tools on demand",
82
+
"Leave MCP tool definitions out of each model call until the bot loads them with load_tools; the system prompt lists their names. Saves the definitions' tokens (tens of thousands for large servers) on every call."),
81
83
"context_trigger_tokens": Tunable("Context", "Clear old tool results above (tokens)",
82
84
"Once a run's messages pass this, tool results from turns before the last two, and their call arguments, are replaced by a placeholder. The system prompt and tool schemas are not counted. Leave empty to derive it from the model's context window (falls back to 40000 when unknown).", 1000),
83
85
"context_clear_at_least": Tunable("Context", "Reclaim at least (tokens)",
0 commit comments