feat(parameters): machine-readable parameter schema per guardrail (#206) - #215
Merged
Conversation
Expose AnyGuardrail.get_parameter_schema(name) -> list[ParameterSpec], a public, import-free accessor returning typed create + validate parameters (name, stage, type, required, default, choices, description) for every guardrail, so downstream consumers can auto-generate config UIs without importing model backends. The specs are GENERATED from signatures: scripts/generate_parameter_data.py introspects each guardrail's __init__/validate + docstrings into the stdlib-only src/any_guardrail/_parameter_data.py leaf, which the import-free parameter_registry assembles into frozen ParameterSpecs. enum choices are reused from SUPPORTED_MODELS (model_id), the content registry (criteria/policy/rubric), and the prompt registry (prompt_version). schemas/guardrail_parameters.json is the external JSON export. Two --check pre-commit hooks fail on drift. Part of #206. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Assert the committed _parameter_data.py and schemas/guardrail_parameters.json match fresh generation (no drift), validate-stage specs partition the taxonomy's required/optional_validate_kwargs, model_id enum choices equal SUPPORTED_MODELS, every enum spec carries choices, parameters.py/_parameter_data.py stay stdlib+pydantic leaves, and get_parameter_schema imports no guardrail implementation module. Part of #206. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Document AnyGuardrail.get_parameter_schema, the generate_parameter_data.py / generate_parameters_json.py regeneration commands, and schemas/guardrail_parameters.json in CLAUDE.md. Part of #206. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
Adds a new, import-free way to discover each guardrail’s configurable parameters (both create/__init__ and extra validate kwargs) via a typed ParameterSpec schema, plus deterministic generated artifacts and parity tests to prevent drift.
Changes:
- Introduces
ParameterSpec/ParameterStage/ParameterTypeand an import-freePARAMETER_REGISTRYwithAnyGuardrail.get_parameter_schema(...). - Adds generators to derive parameter data from guardrail signatures/docstrings and export
schemas/guardrail_parameters.json, with pre-commit--checkhooks. - Adds unit tests enforcing drift-free generation, enum/choices invariants, and “no guardrail imports” during schema access.
Reviewed changes
Copilot reviewed 13 out of 13 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/unit/test_parameters.py | Adds parity/invariant tests for the generated parameter schema and leaf-module guarantees. |
| src/any_guardrail/types.py | Re-exports parameter schema types from any_guardrail.types. |
| src/any_guardrail/parameters.py | Defines leaf Pydantic models/enums for parameter schema (ParameterSpec, etc.). |
| src/any_guardrail/parameter_registry.py | Adds import-free registry assembly + accessor for per-guardrail parameter specs. |
| src/any_guardrail/api.py | Exposes public AnyGuardrail.get_parameter_schema(...) API surface. |
| src/any_guardrail/_parameter_data.py | Adds generated, stdlib-only embedded JSON payload backing the registry. |
| src/any_guardrail/init.py | Re-exports parameter schema types at package top level. |
| scripts/generate_parameters_json.py | Adds generator for schemas/guardrail_parameters.json export. |
| scripts/generate_parameter_data.py | Adds signature/docstring introspection generator for _parameter_data.py. |
| schemas/guardrail_parameters.json | Adds committed JSON export of per-guardrail parameter specs. |
| pyproject.toml | Updates mypy overrides for new script modules imported in tests. |
| CLAUDE.md | Documents new regeneration commands for parameter schema artifacts. |
| .pre-commit-config.yaml | Adds --check hooks to prevent drift of generated parameter artifacts. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- get_parameter_schema indexes PARAMETER_REGISTRY directly instead of .get(), so a missing entry (an internal invariant violation) fails fast with KeyError instead of masquerading as 'no parameters'; zero-param guardrails still return [] via their empty-tuple entry. - generate_parameter_data.py main() creates the output parent directory before writing, matching the other generators (so --out to a new dir works). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a public, import-free
AnyGuardrail.get_parameter_schema(name) -> list[ParameterSpec]so downstream consumers (config-UI builders) can discover, type, and validate each guardrail's parameters programmatically instead of hand-writing JSON blobs.Closes #206.
What it returns
One
ParameterSpecper parameter, covering bothcreate(__init__) andvalidateparams:namestagecreate|validatetypestring|integer|number|boolean|enum|jsonrequireddefaultchoicesenum(e.g.model_id→SUPPORTED_MODELS)descriptionjsonis the explicit "not flat-form-able, use a JSON editor" signal (nested dict / list-of-dict). Guardrails with no params return[].Design — generated from signatures (per the issue's option)
scripts/generate_parameter_data.pyintrospects every guardrail's__init__/validatesignatures + docstrings into the stdlib-only leafsrc/any_guardrail/_parameter_data.py(payload embedded as a parsed JSON string, so the module is byte-for-byte reproducible).parameter_registry.py(import-free) assembles that leaf into frozenParameterSpecs. The accessor reads it without importing any guardrail implementation ortorch/transformers— works in a bare install.enumchoices reuse existing sources:SUPPORTED_MODELS(model_id), the content registrylist_criteria/list_policies/list_rubrics(criteria/policy/rubric), and the prompt registrylist_prompt_versions(prompt_version).schemas/guardrail_parameters.jsonis the external export (likeguardrail_prompts.json); two--checkpre-commit hooks fail on drift.provider,api_key) are excluded; the primaryvalidateinput text is excluded (validate specs cover the extra kwargs).Verification
pytest tests/unit/test_parameters.py— 162 passed. Invariants enforced: committed data + JSON match fresh generation (no drift); validate-stage specs partition each guardrail'srequired/optional_validate_kwargs(reconciled with the taxonomy);model_idenum choices ==SUPPORTED_MODELS; everyenumcarries choices;parameters.py/_parameter_data.pystay stdlib+pydantic leaves;get_parameter_schemaimports no guardrail module (subprocess check).pytest tests/unit— 844 passed.pre-commit run— ruff, ruff-format, mypy (strict), and the two newguardrail-parameter-data/guardrail-parameters-json--checkhooks all pass.Work required outside this repo
None — self-contained. (The generator introspects installed classes; regeneration is wired into pre-commit and CLAUDE.md documents it, so adding/changing a guardrail's params needs no hand-maintained list.)
🤖 Generated with Claude Code