Skip to content

Commit 88b9328

Browse files
feat(ui): add Beta dynamic tools toggle to Agent UI settings (#1798) (#1857)
## Summary The semantic dynamic tool loader — which cuts first-turn latency on the Doc Agent by trimming each turn's tool prompt to a semantically-matched subset — has shipped **dark** since #1449: the only way to switch it on was exporting `GAIA_DYNAMIC_TOOLS` or editing SDK config, so no Agent UI user could discover or try it. This PR adds a single **Beta** toggle under **Settings → Dynamic Tools** so users can opt in without touching env vars. It stays **off by default** and changes *no* loader behavior — it only controls whether the UI-built agent sets `config.dynamic_tools=true`. ## Why The feature was complete but undiscoverable from the UI — its only on-ramps were a dev/CI env var and an SDK field. `GAIA_DYNAMIC_TOOLS` must stay the authoritative dev/CI/eval override (eval tooling and CI depend on it), so the toggle layers on top rather than replacing it: when the env var is set it **wins**, and the toggle then reflects the effective value and disables itself with an explanation — never a silent no-op that lies about the running state. ## Linked issue Closes #1798 ## Changes - **Settings toggle (Beta, off by default).** Boolean toggle reusing the existing `.toggle-switch` pattern with an immediate optimistic save; on failure it reverts and surfaces a visible error (no silent fallback). When `GAIA_DYNAMIC_TOOLS` is set, the toggle shows the effective value, disables, and explains why. - **Env-wins precedence centralized in one parser.** Extracted `dynamic_tools_env_override()` so the agent resolver and the settings router parse the *same* truthy set — no drift between what the toggle shows and what the agent does. `GET`/`PUT /api/settings` now return `dynamic_tools` + `dynamic_tools_locked`; `PUT` persists the user's intent even while the env var locks the effective value, so their choice applies once it's unset. - **Threaded through every agent construction path, not just the active one.** `_session_agent_kwargs` carries the field to both the streaming and non-streaming Doc-agent builds (the only path where the loader is observable). The scheduled-prompt build and the autonomous agent-loop tick also read it — inert on their `"full"` profile today, wired and commented so they can't silently diverge if that ever changes. - **Docs + polish (review follow-ups).** Documented the toggle as a third enable path in the chat guide; added a disabled-state style to the shared toggle so a locked control *looks* locked; added a double-click-during-save guard test. <details> <summary>Deviations from the issue sketch (flagged per plan)</summary> | Issue/sketch assumed | Code reality | Resolution in this PR | |---|---|---| | Wire at the two `_chat_helpers.py` `chat` sites | Those build the default `"full"` profile, where the loader is inert; the loader-active path is the **`doc`** agent built via `registry.create_agent` in the `else` branch | Thread `dynamic_tools` through `_session_agent_kwargs`, which reaches the active `doc` path **and** the inert `chat` sites in one place | | "thread through `_session_agent_kwargs` covers all sites" | `server.py`'s scheduled-prompt path does **not** call that helper | `server.py` gets its own `db.get_setting("dynamic_tools")` read + an inert-on-`"full"` comment | | Frontend should mirror `custom_model` (text input + Save button) | The new control is **boolean** | Used the `.toggle-switch` pattern (immediate save) — appropriate for a boolean, not a text+Save field | | Persist "mirroring `agent_mode`" | Settings persist as TEXT; booleans use `"true"`/`"false"` (precedent: `memory_enabled`) | Persist `"true"`/`"false"`; read with the same string compare; missing → off | | TS `Settings` mirrors `SettingsResponse` | TS `Settings` omits `agent_mode` (backend-only) | Added `dynamic_tools` + `dynamic_tools_locked` to the TS type only | | Plan listed only `server.py` for the inert future-proofed read | The autonomous **`agent_loop.py`** tick is a separate `ChatAgentConfig` build site with the same concern | Also wired + commented there, so both background paths stay in lock-step | **No eval run** — deliberate, not skipped. The default-off path is byte-identical to today (`config.dynamic_tools is False` ⇒ loader `None`); this PR adds a UI surface + plumbing that sets an already-eval-covered config field and touches no prompt, tool registration, or selection logic. Loader behavior remains covered by #1449/#1762's committed `scorecard_tool_selection.json`. </details> ## Test plan - [x] **Backend unit** — round-trip, env-lock, persist-while-locked, cold-state default-off, per-path contract-shape, and the env-override helper: ```bash python -m pytest tests/unit/chat/ui/test_server.py \ tests/unit/chat/ui/test_chat_helpers_model_resolution.py \ tests/unit/test_chat_dynamic_tools.py -xvs ``` - [x] **Frontend unit** — off/on, visible-label click, env-locked-disabled, error-revert, and single-PUT-on-double-click (7 cases): ```bash cd src/gaia/apps/webui && npx vitest run src/components/__tests__/SettingsPage.test.tsx ``` - [x] **Lint** — `python util/lint.py --all` → clean (black/isort/ruff). - [x] **Cold/empty state** — against a fresh UI DB (no `dynamic_tools` key), `GET /api/settings` returns `dynamic_tools: false, dynamic_tools_locked: false`. - [x] **(Optional, on-hardware)** `gaia chat --ui` → Settings → toggle **Dynamic Tools** on → start a **Doc Agent** session → confirm one `TOOL_LOADER {…}` INFO line in the server log (off ⇒ none). With `GAIA_DYNAMIC_TOOLS=1` exported, the toggle renders on **and** disabled. ## Checklist - [x] I have linked a GitHub issue above (`Closes #1798`). - [x] I have described **why** this change is being made, not just what changed. - [x] I have run linting and tests locally (`python util/lint.py --all`, `pytest tests/unit/`). - [x] I have updated documentation if user-visible behavior changed (`docs/guides/chat.mdx` — the toggle as a third enable path). --------- Co-authored-by: Alexey Tyurin <>
1 parent 380cbb5 commit 88b9328

16 files changed

Lines changed: 535 additions & 9 deletions

File tree

docs/guides/chat.mdx

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -316,8 +316,10 @@ first-turn prompt and speeds up the first reply.
316316

317317
It activates only on the `doc` profile (the registered `doc` agent, the SDK with
318318
`ChatAgentConfig(prompt_profile="doc")`, or `gaia eval agent --agent-type doc`).
319-
Turn it on with the config field or an environment variable — the env var wins,
320-
which is handy for the eval harness:
319+
Turn it on with the config field, an environment variable, or the Agent UI
320+
**Settings → Dynamic Tools (Beta)** toggle. The env var wins over both — when it
321+
is set, the UI toggle reflects the effective value and disables itself — which is
322+
handy for the eval harness:
321323

322324
```python
323325
from gaia.agents.chat.agent import ChatAgent, ChatAgentConfig

docs/plans/tool-loader.mdx

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -266,6 +266,11 @@ A malformed `*_TAU` / `*_MAX` value raises at construction (no silent default).
266266
The loader is built **only** for `prompt_profile == "doc"` with the toggle on;
267267
otherwise `self.tool_loader is None` and the agent stays on the legacy path.
268268

269+
As of #1798 the `dynamic_tools` enable knob is also reachable as a **Beta** toggle
270+
in the Agent UI **Settings** panel (default off). `GAIA_DYNAMIC_TOOLS` still wins
271+
when set — the toggle then reflects the effective value and disables — so τ and
272+
the cap stay env-only tuning.
273+
269274
**CORE (10, always-on, cap- & eviction-exempt)** — defined in
270275
[`tool_bundles.py`](https://github.com/amd/gaia/blob/main/src/gaia/agents/chat/tool_bundles.py):
271276
`remember`, `recall`, `update_memory`, `forget`, `search_past_conversations`,

src/gaia/agents/chat/agent.py

Lines changed: 19 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -46,6 +46,21 @@
4646
logger = get_logger(__name__)
4747

4848

49+
def dynamic_tools_env_override() -> Optional[bool]:
50+
"""Parse the ``GAIA_DYNAMIC_TOOLS`` override, or ``None`` when it is unset.
51+
52+
Returns the parsed boolean (truthy set ``1``/``true``/``yes``/``on``,
53+
case-insensitive) when the env var is set, else ``None`` to signal "no
54+
override — fall back to the persisted/config value". The UI settings
55+
router reuses this so the env-wins precedence and the truthy set never
56+
drift between the agent resolver and the toggle that surfaces it.
57+
"""
58+
raw = os.getenv("GAIA_DYNAMIC_TOOLS")
59+
if raw is None:
60+
return None
61+
return raw.strip().lower() in ("1", "true", "yes", "on")
62+
63+
4964
@dataclass
5065
class ChatAgentConfig:
5166
"""Configuration for ChatAgent."""
@@ -460,10 +475,10 @@ def _maybe_build_tool_loader(self) -> Optional[ToolLoader]:
460475

461476
def _resolve_dynamic_tools_enabled(self) -> bool:
462477
"""Toggle: ``GAIA_DYNAMIC_TOOLS`` (truthy) wins over the config field."""
463-
raw = os.getenv("GAIA_DYNAMIC_TOOLS")
464-
if raw is None:
465-
return bool(self.config.dynamic_tools)
466-
return raw.strip().lower() in ("1", "true", "yes", "on")
478+
override = dynamic_tools_env_override()
479+
if override is not None:
480+
return override
481+
return bool(self.config.dynamic_tools)
467482

468483
def _resolve_dynamic_tools_threshold(self) -> float:
469484
"""Threshold: ``GAIA_DYNAMIC_TOOLS_TAU`` wins; malformed value fails loudly."""

src/gaia/apps/webui/src/components/ConnectorsSection.css

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -390,6 +390,11 @@
390390
outline-offset: 2px;
391391
}
392392

393+
.toggle-switch input:disabled + .toggle-track {
394+
opacity: 0.5;
395+
cursor: not-allowed;
396+
}
397+
393398
.grant-scope-warning {
394399
display: flex;
395400
align-items: center;

src/gaia/apps/webui/src/components/SettingsPage.tsx

Lines changed: 72 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,12 @@ export function SettingsPage() {
3434
const [justSaved, setJustSaved] = useState(false);
3535
const justSavedTimerRef = useRef<ReturnType<typeof setTimeout> | null>(null);
3636

37+
// Dynamic Tools (Beta) toggle — #1798
38+
const [dynamicTools, setDynamicTools] = useState(false);
39+
const [dynamicToolsLocked, setDynamicToolsLocked] = useState(false);
40+
const [savingDynamicTools, setSavingDynamicTools] = useState(false);
41+
const [dynamicToolsError, setDynamicToolsError] = useState<string | null>(null);
42+
3743
useEffect(() => {
3844
log.system.info('Checking system status...');
3945
const t = log.system.time();
@@ -66,6 +72,8 @@ export function SettingsPage() {
6672
const value = s.custom_model ?? '';
6773
setCustomModel(value);
6874
setSavedCustomModel(value);
75+
setDynamicTools(s.dynamic_tools);
76+
setDynamicToolsLocked(s.dynamic_tools_locked);
6977
})
7078
.catch((err) => {
7179
log.system.error('Failed to load settings', err);
@@ -101,6 +109,30 @@ export function SettingsPage() {
101109
}
102110
}, [customModel]);
103111

112+
const toggleDynamicTools = useCallback(async () => {
113+
if (dynamicToolsLocked || savingDynamicTools) return;
114+
const previous = dynamicTools;
115+
const next = !previous;
116+
setDynamicTools(next); // optimistic
117+
setSavingDynamicTools(true);
118+
setDynamicToolsError(null);
119+
try {
120+
log.system.info('Saving dynamic_tools setting', { dynamic_tools: next });
121+
const updated = await api.updateSettings({ dynamic_tools: next });
122+
// Trust the server's effective value (env override may win).
123+
setDynamicTools(updated.dynamic_tools);
124+
setDynamicToolsLocked(updated.dynamic_tools_locked);
125+
} catch (err) {
126+
// No silent fallback: revert the optimistic flip and surface the error.
127+
const msg = err instanceof Error ? err.message : String(err);
128+
log.system.error('Failed to save dynamic_tools', err);
129+
setDynamicTools(previous);
130+
setDynamicToolsError(msg);
131+
} finally {
132+
setSavingDynamicTools(false);
133+
}
134+
}, [dynamicTools, dynamicToolsLocked, savingDynamicTools]);
135+
104136
const customModelDirty = customModel.trim() !== savedCustomModel.trim();
105137

106138
const modelName = status?.default_model_name ?? DEFAULT_MODEL_NAME;
@@ -433,6 +465,46 @@ export function SettingsPage() {
433465
</p>
434466
</section>
435467

468+
{/* Dynamic Tools (Beta) — #1798 */}
469+
<section className="settings-section">
470+
<h4>Dynamic Tools <span className="beta-badge">BETA</span></h4>
471+
<p className="model-override-desc">
472+
Trim each turn&rsquo;s tool list to a semantically-matched subset to
473+
speed up the first response. Currently affects the Doc Agent only.
474+
</p>
475+
{/* <label> wraps the row so a click on the text or the track
476+
forwards to the visually-hidden checkbox (matches the
477+
working connector/grant toggles). */}
478+
<label className="setting-row">
479+
<span>Enable dynamic tool loading</span>
480+
<span className="toggle-switch">
481+
<input
482+
type="checkbox"
483+
checked={dynamicTools}
484+
onChange={() => void toggleDynamicTools()}
485+
disabled={!settingsLoaded || savingDynamicTools || dynamicToolsLocked}
486+
aria-label={dynamicTools ? 'Disable dynamic tools' : 'Enable dynamic tools'}
487+
/>
488+
<span className="toggle-track" />
489+
</span>
490+
</label>
491+
{dynamicToolsLocked && (
492+
<p className="model-status-hint">
493+
Controlled by <code>GAIA_DYNAMIC_TOOLS</code> &mdash; unset that
494+
environment variable to change this here.
495+
</p>
496+
)}
497+
{dynamicToolsError && (
498+
<div className="model-warning" role="alert">
499+
<AlertCircle size={14} />
500+
<div className="model-warning-content">
501+
<strong>Could not save</strong>
502+
<p>{dynamicToolsError}</p>
503+
</div>
504+
</div>
505+
)}
506+
</section>
507+
436508
{/* Memory Warnings */}
437509
{status && status.memory_available_gb != null && (() => {
438510
const available = status.memory_available_gb;
Lines changed: 140 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,140 @@
1+
// Copyright(C) 2025-2026 Advanced Micro Devices, Inc. All rights reserved.
2+
// SPDX-License-Identifier: MIT
3+
4+
import { render, screen, waitFor } from '@testing-library/react';
5+
import userEvent from '@testing-library/user-event';
6+
import { beforeEach, describe, expect, it, vi } from 'vitest';
7+
import { SettingsPage } from '../SettingsPage';
8+
import { useChatStore } from '../../stores/chatStore';
9+
import type { Settings } from '../../types';
10+
import * as api from '../../services/api';
11+
12+
vi.mock('../../services/api');
13+
14+
// Heavy child sections make their own API calls; stub them so this test
15+
// stays focused on the Dynamic Tools toggle.
16+
vi.mock('../CustomAgentsSection', () => ({ CustomAgentsSection: () => null }));
17+
vi.mock('../ConnectorsSection', () => ({ ConnectorsSection: () => null }));
18+
19+
const mockedApi = vi.mocked(api);
20+
21+
function makeSettings(overrides: Partial<Settings> = {}): Settings {
22+
return {
23+
custom_model: null,
24+
model_status: null,
25+
context_size: null,
26+
dynamic_tools: false,
27+
dynamic_tools_locked: false,
28+
...overrides,
29+
};
30+
}
31+
32+
beforeEach(() => {
33+
vi.clearAllMocks();
34+
mockedApi.getSystemStatus.mockResolvedValue(null as never);
35+
mockedApi.getMCPRuntimeStatus.mockResolvedValue({ servers: [] } as never);
36+
mockedApi.getSettings.mockResolvedValue(makeSettings());
37+
mockedApi.updateSettings.mockImplementation(async (patch) =>
38+
makeSettings(patch as Partial<Settings>),
39+
);
40+
useChatStore.setState({ sessions: [], agents: [] });
41+
});
42+
43+
function getDynamicToolsToggle(): HTMLInputElement {
44+
// Both enabled/disabled labels resolve to the same control.
45+
return (screen.queryByLabelText('Enable dynamic tools')
46+
?? screen.getByLabelText('Disable dynamic tools')) as HTMLInputElement;
47+
}
48+
49+
describe('SettingsPage — Dynamic Tools toggle (#1798)', () => {
50+
it('renders the loaded value (off by default)', async () => {
51+
render(<SettingsPage />);
52+
const toggle = await waitFor(getDynamicToolsToggle);
53+
expect(toggle).not.toBeChecked();
54+
expect(toggle).not.toBeDisabled();
55+
});
56+
57+
it('reflects a persisted-on value from the server', async () => {
58+
mockedApi.getSettings.mockResolvedValue(makeSettings({ dynamic_tools: true }));
59+
render(<SettingsPage />);
60+
await waitFor(() => expect(getDynamicToolsToggle()).toBeChecked());
61+
});
62+
63+
it('persists dynamic_tools: true when toggled on', async () => {
64+
render(<SettingsPage />);
65+
const toggle = await waitFor(getDynamicToolsToggle);
66+
67+
await userEvent.click(toggle);
68+
69+
await waitFor(() =>
70+
expect(mockedApi.updateSettings).toHaveBeenCalledWith({ dynamic_tools: true }),
71+
);
72+
await waitFor(() => expect(getDynamicToolsToggle()).toBeChecked());
73+
});
74+
75+
it('toggles when the visible row/label is clicked, not just the hidden input', async () => {
76+
// Regression: the checkbox is visually hidden (width:0/height:0) and the
77+
// track is a sibling <span>. Clicking the visible control only works if a
78+
// <label> forwards the click to the input. Clicking the input element
79+
// directly (as the other tests do) would pass even if that wiring is
80+
// missing — so click the visible label text here, the way a user does.
81+
render(<SettingsPage />);
82+
await waitFor(getDynamicToolsToggle);
83+
84+
await userEvent.click(screen.getByText('Enable dynamic tool loading'));
85+
86+
await waitFor(() =>
87+
expect(mockedApi.updateSettings).toHaveBeenCalledWith({ dynamic_tools: true }),
88+
);
89+
await waitFor(() => expect(getDynamicToolsToggle()).toBeChecked());
90+
});
91+
92+
it('is disabled and reflects the env value when locked', async () => {
93+
mockedApi.getSettings.mockResolvedValue(
94+
makeSettings({ dynamic_tools: true, dynamic_tools_locked: true }),
95+
);
96+
render(<SettingsPage />);
97+
const toggle = await waitFor(getDynamicToolsToggle);
98+
99+
expect(toggle).toBeChecked();
100+
expect(toggle).toBeDisabled();
101+
expect(screen.getByText(/GAIA_DYNAMIC_TOOLS/)).toBeInTheDocument();
102+
});
103+
104+
it('guards against a double-click while a save is in flight (single PUT)', async () => {
105+
// Hold the first save open so the toggle stays mid-save between clicks.
106+
let resolveSave!: (value: Settings) => void;
107+
mockedApi.updateSettings.mockImplementationOnce(
108+
() => new Promise<Settings>((resolve) => { resolveSave = resolve; }),
109+
);
110+
111+
render(<SettingsPage />);
112+
const toggle = await waitFor(getDynamicToolsToggle);
113+
114+
// First click starts the save; the toggle disables (savingDynamicTools).
115+
await userEvent.click(toggle);
116+
await waitFor(() => expect(getDynamicToolsToggle()).toBeDisabled());
117+
118+
// Second click while the save is pending must be a no-op, not a second PUT.
119+
await userEvent.click(getDynamicToolsToggle());
120+
121+
// Let the in-flight save settle.
122+
resolveSave(makeSettings({ dynamic_tools: true }));
123+
124+
await waitFor(() => expect(getDynamicToolsToggle()).toBeChecked());
125+
expect(mockedApi.updateSettings).toHaveBeenCalledTimes(1);
126+
});
127+
128+
it('reverts and surfaces an error when the save fails (no silent fallback)', async () => {
129+
mockedApi.updateSettings.mockRejectedValue(new Error('network down'));
130+
render(<SettingsPage />);
131+
const toggle = await waitFor(getDynamicToolsToggle);
132+
133+
await userEvent.click(toggle);
134+
135+
await waitFor(() => expect(screen.getByRole('alert')).toBeInTheDocument());
136+
expect(screen.getByText('network down')).toBeInTheDocument();
137+
// Optimistic flip reverted back to off.
138+
await waitFor(() => expect(getDynamicToolsToggle()).not.toBeChecked());
139+
});
140+
});

src/gaia/apps/webui/src/types/index.ts

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -357,6 +357,10 @@ export interface Settings {
357357
model_status: ModelStatus | null;
358358
/** Persisted context window size override (tokens). null = use default 32768. */
359359
context_size: number | null;
360+
/** Beta dynamic tool loader (#1798). Effective value: GAIA_DYNAMIC_TOOLS wins when set, else the persisted setting. */
361+
dynamic_tools: boolean;
362+
/** True when GAIA_DYNAMIC_TOOLS locks the value — the toggle reflects the effective value and disables. */
363+
dynamic_tools_locked: boolean;
360364
}
361365

362366
/** Status of the GAIA Agent UI MCP server (exposes UI tools to Claude Code etc.). */

0 commit comments

Comments
 (0)