Skip to content

feat(guard): add deterministic policy engine#131

Merged
hasandemirkiran merged 1 commit into
mainfrom
feat/eng-338-deterministic-policy
May 18, 2026
Merged

feat(guard): add deterministic policy engine#131
hasandemirkiran merged 1 commit into
mainfrom
feat/eng-338-deterministic-policy

Conversation

@michiosw
Copy link
Copy Markdown
Contributor

@michiosw michiosw commented May 15, 2026

Summary

This adds the deterministic policy package for the Local Guard LLM Judge launch.

Before this, Guard's deterministic safety checks lived inside risk.DecideRisk, next to normalization, Markov/noop scoring, legacy ask behavior, and the runtime decision shape. That made the launch path hard to reason about.

Now the deterministic layer is isolated in internal/guard/policy:

result := engine.Evaluate(riskEvent, config)

It evaluates a normalized risk.RiskEvent against a curated Kontext rule pack and an active profile: relaxed, balanced, or strict. It returns only allow or deny, plus reviewer/debug metadata like rule ID, category, profile, policy version, reason code, and matched signals.

Why

This gives ENG-325 a clean seam for the launch flow:

hook event
-> risk normalization
-> deterministic policy
-> if deny: stop
-> if allow: local LLM judge
-> final allow/deny

This PR does not wire the package into the running Guard path yet. That stays with ENG-325.

What changed

  • Added internal/guard/policy
  • Added deterministic profiles: relaxed, balanced, strict
  • Added the curated default rule pack
  • Ported current deterministic guard cases into named policy rules
  • Added structured policy.Result diagnostics
  • Added config/default validation helpers
  • Added tests for profile behavior and deny metadata
  • Added risk.IsPersistentResourceClass so policy can reuse risk taxonomy

Verification

  • go test ./internal/guard/policy ./internal/guard/risk ./internal/guard/app/server

Copy link
Copy Markdown
Contributor Author

michiosw commented May 15, 2026

@michiosw michiosw marked this pull request as ready for review May 15, 2026 09:49
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 0e72b298e5

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread internal/guard/policy/engine.go Outdated
@greptile-apps
Copy link
Copy Markdown

greptile-apps Bot commented May 15, 2026

Greptile Summary

This PR introduces a deterministic policy engine (internal/guard/policy) that evaluates risk.RiskEvents against an ordered rule pack and returns an allow or deny Result before any probabilistic scoring stage. It also exposes the previously private isPersistentResource helper as a public function in the risk package.

  • Engine & rule pack: Engine.Evaluate iterates rules in priority order, returns on the first match, and falls back to allow. Five default rules cover credential access, destructive persistent-resource operations, production mutations, direct infra API calls with credentials, and unknown high-risk commands.
  • Profile-gated categories: ProfileRelaxed enables only the two most critical categories; ProfileBalanced and ProfileStrict progressively unlock more, with ProfileStrict adding unknown-command and provider-API-call blocking.
  • Types & config: Config.Validate() enforces known profiles and rule pack IDs; DefaultConfig() sets ProfileBalanced and NonBypassableRules: true.

Confidence Score: 3/5

Safe to merge with the production-mutation predicate fixed; without that fix a directly constructed RiskEvent with an empty OperationClass in a production environment will be incorrectly denied.

The production_mutation rule's When predicate excludes "unknown" and "read" but not the empty string, so any RiskEvent built without an OperationClass and with Environment="production" silently triggers a deny. This is a real false-positive path even if the current normalizer always sets the field; callers constructing RiskEvents directly (including future tests or integrations) will hit it.

internal/guard/policy/pack_default.go — the production_mutation When predicate; internal/guard/policy/types.go — Validate allocation.

Important Files Changed

Filename Overview
internal/guard/policy/pack_default.go Defines the default five-rule pack. The production_mutation rule fires on empty OperationClass (zero value "") because the predicate only excludes "unknown" and "read", creating a false-positive path for directly constructed RiskEvents.
internal/guard/policy/engine.go New deterministic policy engine: evaluates rules in order, returns first match or allow. Value-receiver design means the DefaultRulePack() fallback is re-allocated on every call for a zero-value engine.
internal/guard/policy/types.go Defines all types, constants, Config, and Result. Validate() unnecessarily allocates DefaultRulePack() (with closures) just to compare the ID string; extracting the ID as a constant would fix this.
internal/guard/policy/profiles.go Maps profiles to enabled rule categories. Relaxed profile intentionally only enables the two most critical categories; balanced/strict progressively add more. Clean logic.
internal/guard/policy/rule.go Simple Rule and RulePack struct definitions. No issues.
internal/guard/policy/engine_test.go Comprehensive table-driven tests covering all profiles and categories. Missing a test case for the empty-OperationClass production mutation edge case.
internal/guard/risk/taxonomy.go Thin public wrapper exposing the private isPersistentResource helper to the policy package. Clean single-function file.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[RiskEvent] --> B[Engine.Evaluate]
    B --> C{pack.ID empty?}
    C -- yes --> D[use DefaultRulePack]
    C -- no --> E[use provided pack]
    D --> F[iterate rules]
    E --> F
    F --> G{categoryEnabled for profile?}
    G -- no --> H[skip rule]
    H --> F
    G -- yes --> I{rule.When event ?}
    I -- no --> H
    I -- yes --> J[Result: Deny - fill RuleID, Category, ReasonCode, NonBypassable, MatchedSignals]
    F -- exhausted --> K[Result: Allow - no rule matched]
Loading

Reviews (1): Last reviewed commit: "feat(guard): add deterministic policy en..." | Re-trigger Greptile

Comment thread internal/guard/policy/pack_default.go
Comment thread internal/guard/policy/types.go
Comment thread internal/guard/policy/engine.go
@michiosw michiosw force-pushed the feat/eng-338-deterministic-policy branch from 0e72b29 to 067c32d Compare May 15, 2026 09:56
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 067c32dbb1

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread internal/guard/policy/engine.go Outdated
Comment thread internal/guard/policy/engine.go Outdated
@michiosw michiosw force-pushed the feat/eng-338-deterministic-policy branch 2 times, most recently from db35ad7 to 39da86c Compare May 15, 2026 10:05
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 39da86c5cd

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread internal/guard/policy/types.go
Comment thread internal/guard/policy/profiles.go
@michiosw michiosw force-pushed the feat/eng-338-deterministic-policy branch 2 times, most recently from 1cc4b8e to 0d47862 Compare May 15, 2026 10:17
@michiosw michiosw requested a review from hasandemirkiran May 15, 2026 10:18
Copy link
Copy Markdown
Contributor

@hasandemirkiran hasandemirkiran left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No blocking comments from me. Two small non-blocking comments from the AI reviewer:

  1. Production mutation diagnostics may never include matched_signals.
    pack_default.go declares MatchedSignals: []string{"production", "mutation"}, but engine.go only returns signals already present on risk.RiskEvent.Signals, and the normalizer sets Environment / OperationClass without emitting production or mutation signals. Real production mutation denies will likely have empty matched_signals. I’d either emit those signals during normalization or derive them from structured fields.
  2. Config.RulePack looks like config surface that does not currently do anything meaningful.
    withDefaults() sets empty RulePack to guard-default, making the later if cfg.RulePack == "" branch in Evaluate unreachable, and Validate() rejects every non-default pack even though NewEngine(pack) accepts arbitrary packs. I’d either remove RulePack from Config until there is pack selection, or make validation/evaluation validate against the engine’s pack ID.

@michiosw michiosw force-pushed the feat/eng-338-deterministic-policy branch from 0d47862 to 05bb15b Compare May 16, 2026 11:42
Copy link
Copy Markdown
Contributor

hasandemirkiran commented May 18, 2026

Merge activity

  • May 18, 9:10 AM UTC: A user started a stack merge that includes this pull request via Graphite.
  • May 18, 9:10 AM UTC: Graphite rebased this pull request as part of a merge.
  • May 18, 9:13 AM UTC: @hasandemirkiran merged this pull request with Graphite.

Adds the private deterministic policy package for Guard launch.\n\nThe package keeps normalized event extraction in risk and moves deterministic policy semantics into internal/guard/policy.\n\nIt does not wire runtime behavior yet; ENG-325 will call this from the Local Policy Provider.
@hasandemirkiran hasandemirkiran force-pushed the feat/eng-338-deterministic-policy branch from 05bb15b to 492185c Compare May 18, 2026 09:10
@chatgpt-codex-connector
Copy link
Copy Markdown

Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits.
Credits must be used to enable repository wide code reviews.

@hasandemirkiran hasandemirkiran merged commit 3a0f16e into main May 18, 2026
4 checks passed
@hasandemirkiran hasandemirkiran deleted the feat/eng-338-deterministic-policy branch May 18, 2026 09:13
michiosw pushed a commit that referenced this pull request May 18, 2026
🤖 I have created a release *beep* *boop*
---


## [0.7.0](v0.6.0...v0.7.0) (2026-05-18)


### Features

* add local guard judge contract ([#132](#132)) ([ef6a3cd](ef6a3cd))
* **cli:** make kontext start local-first ([#140](#140)) ([02a34bf](02a34bf))
* **dashboard:** show guard diagnostics ([#156](#156)) ([dcbdf4a](dcbdf4a))
* **guard:** add deterministic policy engine ([#131](#131)) ([3a0f16e](3a0f16e))
* **guard:** add policy config store ([#135](#135)) ([520c3b6](520c3b6))
* **guard:** add policy profile dashboard ([#137](#137)) ([0b0e856](0b0e856))
* **guard:** connect deterministic policy and judge ([#154](#154)) ([25f0d32](25f0d32))
* manage local judge runtime ([#136](#136)) ([09f4fc0](09f4fc0))
* **runtime service:** Guard now starts a Unix-socket localruntime.Service alongside the existing HTTP daemon ([#122](#122)) ([3e87e12](3e87e12))
* **runtime service:** Introduce runtime service and make existing Unix socket more generic ([#121](#121)) ([2bbaa93](2bbaa93))


### Bug Fixes

* **dashboard:** contain command drawer text ([#157](#157)) ([4107e31](4107e31))
* **dashboard:** polish activity summary and log groups ([#153](#153)) ([fc79101](fc79101))
* harden npm dependency resolution ([#114](#114)) ([86eadf5](86eadf5))
* optimize judge fixture category matching ([#146](#146)) ([0bad7ab](0bad7ab))
* **repo:** remove repo-wide codeowners ([#133](#133)) ([73557f0](73557f0))

---
This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants