Skip to content

Commit 2ec7fc7

Browse files
dislovelhlclaudeCopilot
authored
Feat/optional governance layer (#921)
## Changes This pull request introduces a new optional governance layer for GAIA agents, providing action-level governance (ACGS-lite semantics) with extension points for future workflow-level features. The governance system is opt-in and does not affect existing agents unless explicitly enabled. The changes include the addition of a new `gaia.governance` package, a comprehensive example agent demonstrating governance features, and detailed documentation to guide users. The governance framework is modular, allowing developers to mix in governance capabilities, tag tools with risk levels, and configure policy engines, reviewers, and audit logging. The most important changes are: **New Governance Framework:** * Added the `gaia.governance` package, introducing a modular governance layer for GAIA agents. This includes the `GovernedAgentMixin`, `GaiaGovernanceAdapter`, risk tagging decorators, and extension points for policy engines, receipt services, and checkpoint runtimes. * Implemented the `GaiaGovernanceAdapter` class, which composes policy evaluation, checkpointing, receipt issuance, and policy version binding into a single entry point. It ensures secure, auditable, and extensible governance flows for agent tool calls. * Provided an `action_mapper` utility to map GAIA tool calls into governance action requests, standardizing how actions are represented for policy evaluation. **Documentation and Examples:** * Added a comprehensive `README.md` for the `gaia.governance` package, including quick start instructions, configuration options, security properties, and extension points. This documentation enables developers to quickly understand and adopt the governance system. * Introduced a new example, `examples/governed_weather_agent.py`, demonstrating how to wrap an agent with governance, define risk-tagged tools, and handle governance decisions (ALLOW, BLOCK, REVIEW) with local and MCP tools. **Packaging:** * Updated `setup.py` to include the new `gaia.governance` package in the distribution, ensuring it is installed and available for import. --- ## Hardening & Polish (added in 4 follow-up commits) Triggered by a PR-review pass that surfaced merge blockers and architectural feedback. All concerns addressed without expanding feature scope. **Merge blockers fixed** — `f242e28 fix(governance): harden error handling and align docs with additive tags` * Tightened five `except Exception` sites that were silently swallowing errors. The most important one (`_resolve_canonical_tool_name`) now logs unexpected resolver errors with `exc_info=True` instead of falling through silently. This closes the alias-bypass risk where governance could check tags on the wrong key when the resolver had a bug. The other four sites (`_lookup_tool_fn`, `_invoke_callback`, `_prompt_review`, `JsonlReceiptService._read_all`) now use specific exception types and log at WARNING. * `_prompt_review` now returns `(approved, exception_or_None)` so `_handle_review_checkpoint` can stamp the exception type and message into the receipt's `metadata.evidence.resolution.reason` (`15bc40b`). The audit log can now distinguish "reviewer chose no" from "reviewer crashed" — previously both produced the same boilerplate `"reviewer rejected"` reason. * Documentation now matches the code: tag merge is **additive (union, deduplicated)** — *not* "explicit dict wins". Updated README, the `@govern` decorator's docstring, and the inline comment in `mixin._build_action_request` to describe what the tests have always asserted. * `_canonical_hash` for BLOCK-receipt evidence now handles non-JSON tool args, complex types, and cycles without falling back to `repr()`, keeping receipts deterministically hashable across all inputs. * `JsonlReceiptService.issue_receipt` now performs strict canonical JSON validation at issue time, rejecting non-canonical metadata (NaN/Inf, opaque objects) so tampered or unparseable receipts cannot land in the audit log. * Public docs registered: new `docs/sdk/sdks/governance.mdx` plus an entry in `docs/docs.json` SDK navigation. Closes the missing-docs blocker. **CI guard** — `2ed500d ci(test_api): cap job runtime at 30 minutes` * The API Tests job had no `timeout-minutes` and was hanging for 4+ hours on the in-flight CI run for this PR. Added a 30-minute cap (covers worst-case Lemonade boot + model pull + tests) so future runs fail fast on hangs. **Polish** — `ca941a9 refactor(governance): polish pass — drop dead code, tighten lock, deep-copy tags` Driven by a parallel three-agent review (code-reviewer + architecture-reviewer + test-engineer): * Deleted `workflow_mapper.py` and `StaticPolicyBindingService.bind_receipt`. Both were "forward-compat seams" with zero callers in src/, tests/, examples/, or docs/. They'll come back in the PR that adds the real event surface, when the actual signature is known. YAGNI. * Tightened `JsonlReceiptService.get_receipt`: cache reads/writes were unsynchronized while a concurrent `issue_receipt` was mutating the same dict under `_lock`. Both paths are now under the lock. * `GovernedAgentMixin.__init__` now deep-copies inner risk-tag lists so a caller cannot mutate the agent's tag table after construction by holding onto the original list reference. * Added a comment on the `bool`-before-`int` ordering in `_canonical_json_value` (subclass relationship — without the order, `True` would canonicalize as `1`). * Debug breadcrumb on receipt-log malformed-line skips, so an operator chasing a missing receipt has something to grep. **Test additions** — `5cdfee5 test(governance): cover hardened error paths and fail-closed branches` Added 6 new tests covering branches that had no regression guard: * `test_resolver_unexpected_exception_logs_and_governs_raw_name` — proves a buggy `_resolve_tool_name` raising RuntimeError still triggers governance on the raw name AND emits an operator-visible warning. Future regression where the warning is swapped for a silent fallback fails this test. * `test_resolver_lookup_error_is_silent_and_governs_raw_name` — proves the expected "tool not in registry" case (`LookupError`) is absorbed silently with no log noise. * `test_unknown_transition_outcome_fails_closed` — proves a custom `CheckpointRuntime` returning a status the mixin doesn't know is denied, not let through. * `test_handle_transition_rejects_unknown_decision_type` — same idea at the adapter layer for an unknown `GovernanceDecision.decision`. * `test_read_all_skips_malformed_lines` — proves a corrupt line in the middle of an audit log doesn't block readers from finding subsequent valid records. * Existing callback-exception and reviewer-exception tests gained `caplog` assertions so a future silent-swallow regression is caught. Plus two readability fixes: renamed `test_explicit_dict_overrides_decorated_tags` → `test_explicit_empty_dict_does_not_downgrade_decorator_tags` (the body asserted additive semantics, the old name said the opposite); replaced hardcoded `"test_governance_adapter.SlotOnlyEvidence"` qualname strings with `f"{Cls.__module__}.{Cls.__qualname__}"` so the tests survive a file rename. **Verification (fresh evidence at HEAD `15bc40b`)** * Governance test suite: **67 passed** (was 27 before the polish — added 5 from the in-flight strict-evidence work and 6 from the polish review). * `python util/lint.py --black --isort`: PASS. * No dead code residue: `git grep` of `workflow_mapper`, `map_gaia_event_to_transition`, `bind_receipt` returns zero matches. * Public-import smoke test: `GaiaGovernanceAdapter.default()` constructs with the four expected components. * Broader unit tests (excl. `tests/unit/chat/` which needs the optional `[ui]` extra): **946 passed, 16 skipped** — no regressions introduced. * Upstream merge of `amd:main` (10+ commits including the YAML-manifest-removal refactor `#914`) is incorporated. `_TOOL_REGISTRY` survived that refactor; governance imports remain green. **Items intentionally not in this PR** (deferred for follow-up): * `Agent.__init__` accepting `**kwargs` so multi-mixin composition (`MCPAgent + GovernedAgentMixin + ApiAgent`) doesn't trip on closed signatures — touches `agents/base/agent.py` and is a separate concern. * Public accessor for `_TOOL_REGISTRY` to replace the `gaia.agents.base.tools._TOOL_REGISTRY` private import in `mixin._lookup_tool_fn`. * Extracting `_canonical_hash` and `_canonical_json_value` to a public `gaia.governance.canonical` module so any conforming `ReceiptServiceProtocol` can verify or recompute hashes independently. * `default()` accepting component overrides for `policy_engine`, `receipt_service`, `checkpoint_runtime`, `policy_binding` so third parties can swap engines without forgoing the factory. These are good ideas that expand public API surface and belong in a focused follow-up PR rather than bundled into this merge. --- ## Governance REVIEW + existing confirmation path Follow-up for PR review 4197475871: this PR takes Path A. Governance remains an opt-in policy layer, but REVIEW decisions now reuse GAIA Agent UI confirmation when the active console advertises `blocking_confirmation = True` (`SSEOutputHandler`). An explicit `governance_reviewer` still takes precedence for non-UI or custom approval flows, and default `AgentConsole` remains fail-closed because its confirmation method auto-approves. Regression coverage added: * Blocking-console fallback: governance REVIEW delegates to `console.confirm_tool_execution` only for consoles marked `blocking_confirmation = True`. * Agent UI path: a governance-tagged REVIEW tool with `SSEOutputHandler` emits the existing `permission_request` event and runs only after approval. * Default-console safety: unmarked consoles are not treated as implicit reviewers, preserving fail-closed behavior. --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Co-authored-by: dislovelhl <dislovelhl@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
1 parent d8cf594 commit 2ec7fc7

38 files changed

Lines changed: 3747 additions & 25 deletions

.github/workflows/claude.yml

Lines changed: 7 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -58,6 +58,7 @@ jobs:
5858
# Auto-review new PRs (including forks)
5959
pr-review:
6060
if: |
61+
github.repository == 'amd/gaia' &&
6162
github.event_name == 'pull_request_target' &&
6263
(github.event.pull_request.draft == false ||
6364
contains(github.event.pull_request.labels.*.name, 'ready_for_ci'))
@@ -312,6 +313,7 @@ jobs:
312313
# only reads the PR diff and posts comments (no commits to the branch).
313314
pr-comment:
314315
if: |
316+
github.repository == 'amd/gaia' &&
315317
github.event_name == 'pull_request_review_comment' &&
316318
contains(github.event.comment.body, '@claude') &&
317319
github.event.pull_request.head.repo.full_name == github.repository
@@ -427,9 +429,10 @@ jobs:
427429
# only reads the PR diff and posts comments (no commits to the branch).
428430
issue-handler:
429431
if: |
430-
github.event_name == 'issues' ||
431-
(github.event_name == 'issue_comment' &&
432-
contains(github.event.comment.body, '@claude'))
432+
github.repository == 'amd/gaia' &&
433+
(github.event_name == 'issues' ||
434+
(github.event_name == 'issue_comment' &&
435+
contains(github.event.comment.body, '@claude')))
433436
runs-on: ubuntu-latest
434437
steps:
435438
- name: Checkout repository
@@ -620,6 +623,7 @@ jobs:
620623
# Generate release notes when PyPi workflow completes successfully on a tag
621624
release-notes:
622625
if: |
626+
github.repository == 'amd/gaia' &&
623627
github.event_name == 'workflow_run' &&
624628
github.event.workflow_run.conclusion == 'success' &&
625629
startsWith(github.event.workflow_run.head_branch, 'v')

.github/workflows/test_api.yml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -42,6 +42,7 @@ jobs:
4242
test-api:
4343
name: API Tests
4444
runs-on: ${{ contains(github.event.pull_request.labels.*.name, 'stx-test') && 'stx-test' || 'stx' }}
45+
timeout-minutes: 30
4546
if: github.event_name != 'pull_request' || github.event.pull_request.draft == false || contains(github.event.pull_request.labels.*.name, 'ready_for_ci')
4647

4748
steps:

docs/docs.json

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -157,7 +157,8 @@
157157
"sdk/sdks/mcp",
158158
"sdk/sdks/llm",
159159
"sdk/sdks/vlm",
160-
"sdk/sdks/audio"
160+
"sdk/sdks/audio",
161+
"sdk/sdks/governance"
161162
]
162163
},
163164
{

docs/sdk/sdks/agent-ui.mdx

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -637,6 +637,7 @@ class AttachDocumentRequest(BaseModel):
637637
| `tool_args` | `tool` (string), `args` (object), `detail` (string) | Tool arguments. `args` is the raw arguments dict passed to the tool. `detail` is a formatted human-readable summary of the arguments. |
638638
| `tool_end` | `success` (boolean) | Tool invocation completed. |
639639
| `tool_result` | `title` (string or null), `summary` (string), `success` (boolean), `result_data` (object or null), `command_output` (object or null) | Tool result with structured data. `summary` is a human-readable result. `result_data` contains typed results (see below). `command_output` contains shell command output (see below). |
640+
| `policy_alert` | `tool` (string), `decision` (`"BLOCK"`), `reason` (string), `rule_ids` (string[]), `policy_version` (string), `receipt_id` (string, optional) | Governance policy blocked a tool before execution. No user action is required; use this to show a visible policy refusal instead of treating the denial as a generic tool failure. |
640641

641642
`result_data` variants in `tool_result`:
642643
- **File list:** `{"type": "file_list", "files": [...], "total": int}` -- up to 20 file entries

docs/sdk/sdks/governance.mdx

Lines changed: 165 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,165 @@
1+
---
2+
title: "Governance: Optional Policy Layer for Agents"
3+
---
4+
5+
<Info>
6+
**Source Code:** [`src/gaia/governance/`](https://github.com/amd/gaia/blob/main/src/gaia/governance/)
7+
</Info>
8+
9+
The governance layer is an **opt-in** module that intercepts every tool call and
10+
applies a policy decision (ALLOW / BLOCK / REVIEW) before the tool runs. It adds
11+
zero overhead when not activated.
12+
13+
## Quick start
14+
15+
```python
16+
from gaia import Agent, tool
17+
from gaia.governance import GaiaGovernanceAdapter, GovernedAgentMixin, govern
18+
19+
20+
@tool
21+
@govern(risk="blocked", reason="destructive")
22+
def wipe_disk() -> dict:
23+
return {"status": "ok"}
24+
25+
26+
class MyAgent(GovernedAgentMixin, Agent):
27+
...
28+
29+
30+
agent = MyAgent(governance_adapter=GaiaGovernanceAdapter.default())
31+
```
32+
33+
When the model calls `wipe_disk`, governance short-circuits the call,
34+
issues a signed receipt to `receipts.jsonl`, and returns a denied result.
35+
36+
## Decision outcomes
37+
38+
| Decision | Effect |
39+
|---|---|
40+
| `ALLOW` | Tool runs as usual. |
41+
| `BLOCK` | Tool is refused. A receipt is written with the full evidence envelope. |
42+
| `REVIEW` | A checkpoint is opened. Governance calls your `governance_reviewer` callback, or Agent UI's blocking confirmation modal when that is the active console. APPROVE -> tool runs; REJECT -> tool is refused. Either way a receipt is written. |
43+
44+
If `REVIEW` fires and neither a reviewer nor a blocking console is available,
45+
the mixin **fails closed** — the tool is denied without executing.
46+
47+
## Tagging tools
48+
49+
**Decorator style** (colocates policy with the tool):
50+
51+
```python
52+
@tool
53+
@govern(risk="review", reason="sends money")
54+
def transfer(amount: float): ...
55+
```
56+
57+
**Dict style** (centralizes policy on the agent):
58+
59+
```python
60+
agent = MyAgent(
61+
governance_adapter=GaiaGovernanceAdapter.default(),
62+
governance_risk_tags={"transfer": ["review"]},
63+
)
64+
```
65+
66+
Tags from both sources are **additive** (union, deduplicated): decorator tags come
67+
first, then dict tags are appended. A tool with `"review"` from a decorator and
68+
`"blocked"` from the dict will carry both tags.
69+
70+
## Configuration
71+
72+
```python
73+
from gaia.governance import GovernanceConfig
74+
75+
# Structured config object
76+
agent = MyAgent(governance=GovernanceConfig(
77+
adapter=GaiaGovernanceAdapter.default(),
78+
actor_id="alice",
79+
workflow_id="session-42",
80+
risk_tags={"delete_record": ["blocked"]},
81+
reviewer=my_reviewer,
82+
))
83+
84+
# Individual kwargs (equivalent)
85+
agent = MyAgent(
86+
governance_adapter=GaiaGovernanceAdapter.default(),
87+
governance_actor_id="alice",
88+
governance_risk_tags={"delete_record": ["blocked"]},
89+
governance_reviewer=my_reviewer,
90+
)
91+
```
92+
93+
## Reviewers
94+
95+
```python
96+
def my_reviewer(tool_name, tool_args, decision) -> bool:
97+
return input(f"approve {tool_name}? [y/N]: ") == "y"
98+
99+
agent = MyAgent(
100+
governance_adapter=GaiaGovernanceAdapter.default(),
101+
governance_reviewer=my_reviewer,
102+
)
103+
```
104+
105+
An explicit `governance_reviewer` takes precedence. If none is configured,
106+
governance delegates to `console.confirm_tool_execution` only when the console
107+
advertises `blocking_confirmation = True`; Agent UI's `SSEOutputHandler` does
108+
this and emits the existing `permission_request` modal. GAIA's default console is
109+
not consulted because its confirmation method auto-approves.
110+
111+
When a policy returns `BLOCK`, the governed tool body is not executed and the
112+
adapter writes a BLOCK receipt. If the active console supports
113+
`print_policy_alert`, GAIA also emits a user-visible policy alert. Agent UI's
114+
`SSEOutputHandler` sends this as a `policy_alert` SSE event with the blocked
115+
tool, decision, reason, rule IDs, policy version, and receipt ID.
116+
117+
## Observability callbacks
118+
119+
```python
120+
def on_decision(tool_name, tool_args, action, decision):
121+
print(f"{tool_name}: {decision.decision}")
122+
123+
agent = MyAgent(
124+
governance_adapter=GaiaGovernanceAdapter.default(),
125+
governance_callback=on_decision,
126+
)
127+
```
128+
129+
Callback exceptions are logged as warnings and never interrupt tool execution.
130+
131+
## Security properties
132+
133+
- **Canonical name resolution** — governance resolves registered tool names before
134+
checking risk tags, so an LLM cannot bypass a tag on `mcp_time_get_current_time`
135+
by calling the alias `get_current_time`.
136+
- **Envelope-bound receipts** — each receipt's `payload_hash` is a SHA-256 of the
137+
full evidence envelope (action, decision, policy version, constitution hash, actor,
138+
timestamp) in strict canonical JSON. Any tampered field changes the hash.
139+
- **Workflow-bound checkpoints** — the adapter refuses to resolve a checkpoint under
140+
a `workflow_id` that differs from the one recorded when the checkpoint was opened.
141+
- **Fail-closed REVIEW** — no reviewer registered means deny.
142+
143+
## Extension points
144+
145+
| Interface | Shipped reference | Swap with |
146+
|---|---|---|
147+
| `PolicyEngine` | `RuleBasedPolicyEngine` | ACGS-lite, LLM judge, OPA |
148+
| `CheckpointRuntime` | `InMemoryCheckpointBridge` | constitutional-swarm checkpoint service |
149+
| `ReceiptServiceProtocol` | `InMemoryReceiptService` / `JsonlReceiptService` | DB, log forwarder, chain anchor |
150+
| `PolicyBindingProtocol` | `StaticPolicyBindingService` | constitutional-swarm policy control plane |
151+
152+
All four are `@runtime_checkable` Protocols — no inheritance required.
153+
154+
## Audit log
155+
156+
`JsonlReceiptService` writes one JSON object per line to a path you choose
157+
(`receipts.jsonl` by default). The log survives process exit and is trivially
158+
`grep`-able:
159+
160+
```bash
161+
grep '"decision":"BLOCK"' receipts.jsonl | jq .
162+
```
163+
164+
For multi-process deployments, replace `JsonlReceiptService` with a dedicated log
165+
forwarder or database-backed receipt service.

0 commit comments

Comments
 (0)