Language: English | 日本語
Summary. Every command that modifies an agent's skills, rules, identity, or constitution must present its probabilistic output to a human and wait for explicit approval before writing. No
--autoflag. Intermediate artifacts (append-only logs, read-only indexes) are exempt.
accepted
2026-03-26 (decided) / 2026-04-19 (re-expressed in this repository)
Self-improving agents periodically distill episode history into higher-layer artifacts: behavioral skills, operational rules, identity text, and — at the top — a constitution. Each of these layers directly changes how the agent acts in future sessions.
Two common patterns fail to keep a human reliably in the loop:
- Preview-then-run: the workflow is "run
--dry-run, inspect output, then re-run without--dry-run." This breaks because LLM output is stochastic — the preview is not the same text as the production run. What the human approved is not what gets written. - Advisory confirmation: an optional flag that can be skipped. Any automation layer (a cron job, a supervising agent, an impatient operator) will eventually skip it. Human-in-the-loop is not structurally enforced.
The thesis of this project (see ../thesis.md) is that
judgment about what to constrain and who is responsible must persist
across implementation changes. Behavior-modifying writes are exactly where
that judgment applies. Without a structural gate, the decision to change the
agent's identity or rules is delegated to the LLM's probabilistic output.
Commands that write to behavior-modifying artifacts must present the
generated output to a human and wait for explicit approval before writing.
No --auto flag is provided.
| Artifact class | Approval gate | Rationale |
|---|---|---|
| Skills (how the agent acts) | Required | Directly alters behavior |
| Rules (operational policies) | Required | Directly alters behavior |
| Identity (who the agent is) | Required | Directly alters behavior |
| Constitution / values | Required | Highest impact |
| Episode log (append-only) | None | Raw record, no behavioral effect |
| Knowledge store (intermediate) | None | Only influences behavior through a subsequent behavior-modifying command, which has its own gate |
CLI invocation
→ generation (LLM)
→ display result (stdout / UI)
→ prompt: "Apply this change? [y/N]"
→ y → write artifact + append audit record
→ N → discard + append audit record (reason: rejected)
The audit record is non-negotiable. Both approved and rejected decisions are logged, with the reason field populated for rejections and out-of-band changes (see below).
The gate cannot be disabled with a flag. If a command is invoked in a non-interactive context (CI, launchd, a supervising agent), the correct answer is not to auto-approve — it is to halt and require an operator.
The two legitimate non-interactive paths are:
- A supervising agent or orchestrator reads the output and decides. Its decision still flows through the same audit record. "Supervising agent" is treated as an operator, not as a bypass.
- A separately-invoked flag that requires an audit reason. For manual
CRUD operations (e.g.
remove-skill), a non-interactive mode is allowed only when a human-provided--reasonstring is attached to the audit entry. The operator is present; only the prompt is skipped.
Any change to a behavior-modifying artifact — including direct filesystem
edits, file removals, or bulk renames — must route through an auditable
CLI entry point that writes the same audit record. A plain rm on a
skill file is an uncovered path and violates the invariant.
- Optional
--autoflag for automation contexts — Rejected. Creates exactly the class of path this decision is meant to close. A supervising agent can read output and respond instead. - Single
--dry-runflag, no approval gate — Rejected. Preview does not match production output for stochastic generation; what the human approved is not what was written. - Approval gate on every command including intermediate writes — Rejected. Non-behavioral intermediate artifacts (the episode log, scheduled knowledge distillation) run periodically and do not need human-by-human approval. Requiring it makes continuous operation impossible without actually improving safety.
- Probabilistic model-based approval (a second "critic" LLM that approves or rejects) — Rejected. This is the restriction-layer failure mode. A probabilistic gate fails open under prompt injection. The point is a human in the loop, structurally.
- Human-in-the-loop is structurally enforced: there is no flag to bypass.
- The non-reproducibility problem of probabilistic generation is resolved — approval is granted to the actual output that will be written, not to a preview.
- Every behavior change has an audit entry. Causal traceability (ADR-0006) has a reliable source.
- CLI tools that modify behavior cannot be used in naive CI pipelines. This is intentional.
- The operator owns the cognitive load of reviewing each generation. Combined with One External Adapter (ADR-0004), this keeps an adult in the room at every decision that changes what the agent does or how it represents itself.
See the agent-observability-patterns
skill repository (see docs/skills/) for the
installable "how" counterpart: read-only instruments and shadow-mode
validation supply the data a named approver signs off from, instead of
approving on intuition.
Lineage. This principle was crystallized in contemplative-agent docs/adr/0012-human-approval-gate.md. The 2026-04-17 note in that ADR — extending the gate to manual CRUD — is included here as part of the decision, not as a later amendment.