Skip to content

feat(parameters): machine-readable parameter schema per guardrail (#206) - #215

Merged
dni138 merged 4 commits into
mainfrom
feat/parameter-schema
Jul 22, 2026
Merged

feat(parameters): machine-readable parameter schema per guardrail (#206)#215
dni138 merged 4 commits into
mainfrom
feat/parameter-schema

Conversation

@dni138

@dni138 dni138 commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds a public, import-free AnyGuardrail.get_parameter_schema(name) -> list[ParameterSpec] so downstream consumers (config-UI builders) can discover, type, and validate each guardrail's parameters programmatically instead of hand-writing JSON blobs.

Closes #206.

What it returns

One ParameterSpec per parameter, covering both create (__init__) and validate params:

field meaning
name parameter name
stage create | validate
type string | integer | number | boolean | enum | json
required no default in the signature
default JSON-native default
choices for enum (e.g. model_idSUPPORTED_MODELS)
description one line, parsed from the docstring

json is the explicit "not flat-form-able, use a JSON editor" signal (nested dict / list-of-dict). Guardrails with no params return [].

Design — generated from signatures (per the issue's option)

  • scripts/generate_parameter_data.py introspects every guardrail's __init__/validate signatures + docstrings into the stdlib-only leaf src/any_guardrail/_parameter_data.py (payload embedded as a parsed JSON string, so the module is byte-for-byte reproducible).
  • parameter_registry.py (import-free) assembles that leaf into frozen ParameterSpecs. The accessor reads it without importing any guardrail implementation or torch/transformers — works in a bare install.
  • enum choices reuse existing sources: SUPPORTED_MODELS (model_id), the content registry list_criteria/list_policies/list_rubrics (criteria/policy/rubric), and the prompt registry list_prompt_versions (prompt_version).
  • schemas/guardrail_parameters.json is the external export (like guardrail_prompts.json); two --check pre-commit hooks fail on drift.
  • Execution plumbing/credentials (provider, api_key) are excluded; the primary validate input text is excluded (validate specs cover the extra kwargs).

Verification

  • pytest tests/unit/test_parameters.py — 162 passed. Invariants enforced: committed data + JSON match fresh generation (no drift); validate-stage specs partition each guardrail's required/optional_validate_kwargs (reconciled with the taxonomy); model_id enum choices == SUPPORTED_MODELS; every enum carries choices; parameters.py/_parameter_data.py stay stdlib+pydantic leaves; get_parameter_schema imports no guardrail module (subprocess check).
  • pytest tests/unit — 844 passed.
  • pre-commit run — ruff, ruff-format, mypy (strict), and the two new guardrail-parameter-data / guardrail-parameters-json --check hooks all pass.
  • Coverage: 39 guardrails, 154 params exported.

Work required outside this repo

None — self-contained. (The generator introspects installed classes; regeneration is wired into pre-commit and CLAUDE.md documents it, so adding/changing a guardrail's params needs no hand-maintained list.)

🤖 Generated with Claude Code

dni138 and others added 3 commits July 22, 2026 16:17
Expose AnyGuardrail.get_parameter_schema(name) -> list[ParameterSpec], a public,
import-free accessor returning typed create + validate parameters (name, stage,
type, required, default, choices, description) for every guardrail, so downstream
consumers can auto-generate config UIs without importing model backends.

The specs are GENERATED from signatures: scripts/generate_parameter_data.py
introspects each guardrail's __init__/validate + docstrings into the stdlib-only
src/any_guardrail/_parameter_data.py leaf, which the import-free parameter_registry
assembles into frozen ParameterSpecs. enum choices are reused from SUPPORTED_MODELS
(model_id), the content registry (criteria/policy/rubric), and the prompt registry
(prompt_version). schemas/guardrail_parameters.json is the external JSON export.
Two --check pre-commit hooks fail on drift.

Part of #206.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Assert the committed _parameter_data.py and schemas/guardrail_parameters.json match
fresh generation (no drift), validate-stage specs partition the taxonomy's
required/optional_validate_kwargs, model_id enum choices equal SUPPORTED_MODELS, every
enum spec carries choices, parameters.py/_parameter_data.py stay stdlib+pydantic leaves,
and get_parameter_schema imports no guardrail implementation module.

Part of #206.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Document AnyGuardrail.get_parameter_schema, the generate_parameter_data.py /
generate_parameters_json.py regeneration commands, and schemas/guardrail_parameters.json
in CLAUDE.md.

Part of #206.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new, import-free way to discover each guardrail’s configurable parameters (both create/__init__ and extra validate kwargs) via a typed ParameterSpec schema, plus deterministic generated artifacts and parity tests to prevent drift.

Changes:

  • Introduces ParameterSpec/ParameterStage/ParameterType and an import-free PARAMETER_REGISTRY with AnyGuardrail.get_parameter_schema(...).
  • Adds generators to derive parameter data from guardrail signatures/docstrings and export schemas/guardrail_parameters.json, with pre-commit --check hooks.
  • Adds unit tests enforcing drift-free generation, enum/choices invariants, and “no guardrail imports” during schema access.

Reviewed changes

Copilot reviewed 13 out of 13 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
tests/unit/test_parameters.py Adds parity/invariant tests for the generated parameter schema and leaf-module guarantees.
src/any_guardrail/types.py Re-exports parameter schema types from any_guardrail.types.
src/any_guardrail/parameters.py Defines leaf Pydantic models/enums for parameter schema (ParameterSpec, etc.).
src/any_guardrail/parameter_registry.py Adds import-free registry assembly + accessor for per-guardrail parameter specs.
src/any_guardrail/api.py Exposes public AnyGuardrail.get_parameter_schema(...) API surface.
src/any_guardrail/_parameter_data.py Adds generated, stdlib-only embedded JSON payload backing the registry.
src/any_guardrail/init.py Re-exports parameter schema types at package top level.
scripts/generate_parameters_json.py Adds generator for schemas/guardrail_parameters.json export.
scripts/generate_parameter_data.py Adds signature/docstring introspection generator for _parameter_data.py.
schemas/guardrail_parameters.json Adds committed JSON export of per-guardrail parameter specs.
pyproject.toml Updates mypy overrides for new script modules imported in tests.
CLAUDE.md Documents new regeneration commands for parameter schema artifacts.
.pre-commit-config.yaml Adds --check hooks to prevent drift of generated parameter artifacts.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/any_guardrail/parameter_registry.py Outdated
Comment thread scripts/generate_parameter_data.py
- get_parameter_schema indexes PARAMETER_REGISTRY directly instead of .get(),
  so a missing entry (an internal invariant violation) fails fast with KeyError
  instead of masquerading as 'no parameters'; zero-param guardrails still return
  [] via their empty-tuple entry.
- generate_parameter_data.py main() creates the output parent directory before
  writing, matching the other generators (so --out to a new dir works).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@dni138
dni138 merged commit 5768495 into main Jul 22, 2026
9 checks passed
@dni138
dni138 deleted the feat/parameter-schema branch July 22, 2026 20:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Expose a machine-readable parameter schema per guardrail (create + validate params)

2 participants