feat(ui): expose dynamic tool loader as a Beta toggle in Agent UI settings

[kovtcharov]

The dynamic tool loader (#1449 + #1762) trims each turn's tool prompt to a semantically-matched subset to cut first-turn latency (TTFT numbers in #1449) — but it ships dark: reachable only via GAIA_DYNAMIC_TOOLS / config.dynamic_tools, with no Agent UI surface. Add a single Beta toggle in Settings so users can opt in without editing env vars. Stays off by default.

Code locations are as of this writing — confirm before editing.

Current state

• Gate: ChatAgent._resolve_dynamic_tools_enabled() (chat/agent.py:463) — GAIA_DYNAMIC_TOOLS (if set) wins, else config.dynamic_tools (default False).
• Only the ChatAgent doc profile consumes it. The feat(agents): tool loader Part 2 — explicit escape hatch + tuning (#1450) #1762 load_tools escape hatch rides the same gate.

Scope (keep it simple)

• One toggle, marked Beta. No threshold/max controls in the UI (leave τ/max as env-var tuning).
• Default OFF.

Implementation

1. Schema — add dynamic_tools: bool to SettingsResponse + SettingsUpdateRequest (ui/models.py).
2. Backend GET/PUT — persist via db.get_setting/set_setting("dynamic_tools", …) in ui/routers/system.py, mirroring agent_mode (only touch when in model_fields_set).
3. Wire setting → agent — custom_model shows the pattern (read via db.get_setting, passed into ChatAgentConfig(...)). Set dynamic_tools at all three construction sites or the toggle silently won't apply on one path:
   • _chat_helpers.py ~L1230 (non-streaming chat)
   • _chat_helpers.py ~L1633 (streaming chat)
   • server.py:373 (scheduled-prompt agent)
     Cleanest: thread it through _session_agent_kwargs so all sites share one source.
4. Frontend — small toggle with a Beta badge in SettingsPage.tsx, mirroring the existing custom_model control (useState + dirty-tracking + api.updateSettings). agent_mode is backend-only, so custom_model is the right precedent.

Precedence (decided — do not remove the env var)

Keep GAIA_DYNAMIC_TOOLS as the dev/CI/eval override; the UI toggle is the user-facing control, env var wins when set — same dual-surface pattern as context_size↔GAIA_CTX_SIZE and memory↔GAIA_MEMORY_DISABLED. Eval tooling (eval/tool_recall.py, tool_cost.py) and CI run headless and depend on the env var. UX nit: when the env override is set, the toggle should reflect/disable rather than silently no-op.

Tests

• Backend: settings round-trip (PUT → GET) + config.dynamic_tools set from the persisted value at each agent-build path.
• Frontend: toggle component test.
• No eval needed — default-off path unchanged; loader behavior already covered by Tool loader — Part 1: selection + dual-path filtering (CORE + semantic) #1449/feat(agents): tool loader Part 2 — explicit escape hatch + tuning (#1450) #1762.

Files: ui/models.py · ui/routers/system.py · ui/_chat_helpers.py · server.py · SettingsPage.tsx(+.css) · tests. Follow-up to #1762.

[kovtcharov]

@alexey-tyurin you built the tool loader (#1449/#1762) so you've got the most context here — any interest in taking this one on too? It's the natural next step to get your TTFT win in front of actual UI users instead of leaving it env-var-only. Totally fine if you'd rather not, just figured I'd offer it to you first. Either way, thanks for the great work on the loader! 🙌

[github-actions]

The plan checks out — all five cited locations are accurate and the env-wins-over-config precedence already lives in _resolve_dynamic_tools_enabled() (chat/agent.py:462), so the UI toggle only needs to populate config.dynamic_tools. Two things worth baking into the implementation so the third site behaves:

🟡 server.py:373 won't be covered by _session_agent_kwargs. The scheduled-prompt path builds ChatAgentConfig(max_steps=5, silent_mode=True, debug=False) inline and never calls the shared helper — threading it through _session_agent_kwargs covers only the two _chat_helpers.py sites. This one needs its own db.get_setting("dynamic_tools") read.

🟢 The loader is doc-profile-only. _maybe_build_tool_loader() returns None unless prompt_profile == "doc" (chat/agent.py:451). The scheduled-prompt agent runs the default "full" profile, so setting dynamic_tools there is inert today — fine to wire for future-proofing, just not something a test can observe as on yet. Worth a code comment so it doesn't read as a bug.

Both are minor; the design is otherwise solid. The env-override-reflects-and-disables UX nit is the right call for the toggle.

[alexey-tyurin]

@kovtcharov — I'd like to take this one; please assign me. And thanks for all the memory-track work you've been sending my way — I've really been enjoying it.

It's the natural follow-up to Part 2 (#1762) and #1449 — the UI surface for the loader those landed — so it builds directly on that work.

Scope as I read it: a single Beta toggle in Settings, default OFF, no τ/max controls (those stay env-var tuning). I'll keep GAIA_DYNAMIC_TOOLS as the dev/CI/eval override with env-winning precedence — same dual-surface pattern as context_size↔GAIA_CTX_SIZE — and handle the UX detail you flagged: when the env override is set, the toggle reflects/disables rather than silently no-opping. On the backend I'll thread dynamic_tools through _session_agent_kwargs so all three construction sites (non-streaming, streaming, scheduled-prompt) share one source rather than setting it three times and missing one. Frontend mirrors the custom_model control.

I'll confirm the current code locations against main before editing (per your note), follow CLAUDE.md, and cover the settings round-trip + per-path config plumbing + the frontend toggle in tests — no eval needed since the default-off path is unchanged.

[alexey-tyurin]

@kovtcharov-amd - One bit of tracking housekeeping: would it make sense to put #1798 under the
Dynamic Tool Loading milestone (#47)? The milestone already tracks Parts 0–3
(#1448–#1451) and the parent #688, and this issue is the UI surface for exactly
that work — the loader landed via #1449 + #1762 but is currently reachable only
through GAIA_DYNAMIC_TOOLS / config.dynamic_tools. The default-off Beta toggle
is what makes the shipped loader actually reachable for users, so filing it
alongside the rest keeps the build-and-expose story in one place and closes the
loop on the milestone.

Flagging it for your call since you own the milestone field — happy to proceed
on the implementation either way.

[alexey-tyurin]

@kovtcharov-amd — PR for this is ready for review: #1857.

It adds the single Beta toggle under Settings → Dynamic Tools, off by default, with GAIA_DYNAMIC_TOOLS still winning when set (the toggle then reflects the effective value and disables). No loader behavior changes — the default-off path stays byte-identical. Backend and frontend unit suites pass locally; the two red Windows checks are a Lemonade-server-startup CI flake, unrelated to the diff.

[alexey-tyurin]

@kovtcharov-amd — both milestone issues now have PRs up for review.

• feat(ui): expose dynamic tool loader as a Beta toggle in Agent UI settings #1798 → feat(ui): add Beta dynamic tools toggle to Agent UI settings (#1798) #1857 (this issue): the Beta Settings → Dynamic Tools toggle, off by default, env var still wins. Ready for review.
• Tool-registry scaling: resolve scratchpad/memory collision via #688 dynamic loading #800 → test(tool-loader): pin #800 doc-profile data-vs-recall disambiguation #1844, opened yesterday (ready-for-review note): pins the doc-profile data-vs-recall disambiguation for the loader.

These are the last two open items in the Dynamic Tool Loading milestone (5 already closed). Once both PRs merge and #800 + #1798 close, the milestone is complete and can be closed out.

[kovtcharov]

@itomek can you please test this feature.