diff --git a/examples/12-api-contract-validation/README.md b/examples/12-api-contract-validation/README.md new file mode 100644 index 00000000..7bda86af --- /dev/null +++ b/examples/12-api-contract-validation/README.md @@ -0,0 +1,125 @@ +# 12 — API Contract Validation + +An end-to-end API contract validation pipeline that uses PR-triggered +validation, scheduled compatibility audits, and a multi-step consumer +update pipeline to catch and resolve breaking API changes. + +## Use Case + +Detect breaking changes in API definitions (OpenAPI, protobuf, GraphQL) +when PRs are opened, run weekly audits for unreleased drift, and +coordinate migration across consumer repositories when intentional +breaking changes are approved. + +AI agents add value beyond static diff tools by reasoning about semantic +intent, analyzing real consumer impact, proposing backward-compatible +alternatives, and generating targeted migration guides. + +## Resources + +| File | Kind | Purpose | +|------|------|---------| +| `credentials-secret.yaml` | Secret | Anthropic API key for the agent | +| `github-token-secret.yaml` | Secret | GitHub token for cloning and PR creation | +| `workspace.yaml` | Workspace | Git repository containing API definitions | +| `agentconfig.yaml` | AgentConfig | Shared instructions for the API validator agent | +| `taskspawner-webhook.yaml` | TaskSpawner | PR-triggered validation via `/validate-api` comment | +| `taskspawner-cron.yaml` | TaskSpawner | Weekly cross-service compatibility audit | +| `pipeline.yaml` | Task (x2) | Consumer impact analysis and migration guide pipeline | + +## Configurations + +### 1. PR-Triggered API Validation (Webhook) + +Listens for `/validate-api` comments on PRs. The agent checks out the PR +branch, runs structural diff tools if available, classifies each change, +and posts a review summarizing findings with backward-compatible +alternatives for any breaking changes. + +### 2. Scheduled Compatibility Audit (Cron) + +Runs every Monday at 8:00 AM UTC. The agent compares the current API +surface against the latest tagged release, identifies unreleased breaking +changes and API hygiene issues, and creates a GitHub issue with findings. + +### 3. Consumer Update Pipeline (Task Dependencies) + +A two-stage pipeline for coordinating migration after an intentional +breaking change is approved: + +``` +identify-consumers (Task) + │ analyzes which consumers use the affected endpoints + │ outputs: migration-plan.md + │ + ▼ +generate-migration-guide (Task, dependsOn: [identify-consumers]) + │ reads migration-plan.md + │ generates step-by-step migration guide with code examples + │ opens a PR with the guide +``` + +## Steps + +1. **Edit the secrets** — replace placeholders in `credentials-secret.yaml` + and `github-token-secret.yaml`. + +2. **Edit `workspace.yaml`** — set your API service repository URL. + +3. **Review `agentconfig.yaml`** — customize the agent instructions for your + API format and conventions. + +4. **Deploy the webhook-triggered validator:** + +```bash +kubectl apply -f credentials-secret.yaml -f github-token-secret.yaml \ + -f workspace.yaml -f agentconfig.yaml -f taskspawner-webhook.yaml +``` + +5. **Deploy the scheduled audit (optional):** + +```bash +kubectl apply -f taskspawner-cron.yaml +``` + +6. **Run the consumer update pipeline (when needed):** + +```bash +kubectl apply -f pipeline.yaml +``` + +7. **Trigger a validation** — comment `/validate-api` on any PR in the + configured repository. + +8. **Watch spawned Tasks:** + +```bash +kubectl get tasks -w +``` + +9. **Cleanup:** + +```bash +kubectl delete -f examples/12-api-contract-validation/ +``` + +## Complementary Issues + +This use case works with current Kelos primitives but would benefit from +proposed API extensions: + +| Issue | Enhancement | Benefit | +|-------|-------------|---------| +| #778 | `filePatterns` filter | Auto-trigger on API file changes instead of manual `/validate-api` | +| #884 | Cross-repo propagation | Create migration PRs directly in consumer repos | +| #842 | `readOnlyWorkspaces` | Read consumer repos without full write access | +| #881 | `contextSources` | Inject previous audit results into validation prompts | + +## Notes + +- The webhook TaskSpawner uses `issue_comment` with `bodyContains` as a + workaround until `filePatterns` filtering (#778) is available. +- The pipeline Tasks share the same `branch` value, which serializes them + automatically in addition to the explicit `dependsOn` ordering. +- Adjust `maxConcurrency` on the webhook TaskSpawner based on your expected + PR volume. diff --git a/examples/12-api-contract-validation/agentconfig.yaml b/examples/12-api-contract-validation/agentconfig.yaml new file mode 100644 index 00000000..9fb3010e --- /dev/null +++ b/examples/12-api-contract-validation/agentconfig.yaml @@ -0,0 +1,25 @@ +apiVersion: kelos.dev/v1alpha1 +kind: AgentConfig +metadata: + name: api-validator-config +spec: + agentsMD: | + # API Contract Validator Agent + + You are an API contract validation specialist. Your role is to analyze API + definition changes for backward compatibility and consumer impact. + + ## Key Conventions + - Always classify changes as non-breaking, potentially-breaking, or breaking + - A "breaking change" means any change that could cause existing consumers + to fail without code modifications + - When suggesting alternatives, prefer deprecation over removal + - Reference the specific API versioning strategy used in this project + - Include concrete code examples in migration guides + + ## Tools Available + - Use `git diff` to compare API definitions between versions + - Use `gh api` to query for dependent repositories or PRs + - Use `gh pr review` to post validation results + - If `oasdiff`, `buf`, or `graphql-inspector` are available, use them first + for structural analysis before adding semantic evaluation diff --git a/examples/12-api-contract-validation/credentials-secret.yaml b/examples/12-api-contract-validation/credentials-secret.yaml new file mode 100644 index 00000000..9ab1464b --- /dev/null +++ b/examples/12-api-contract-validation/credentials-secret.yaml @@ -0,0 +1,8 @@ +apiVersion: v1 +kind: Secret +metadata: + name: anthropic-api-key +type: Opaque +stringData: + # TODO: Replace with your Anthropic API key + ANTHROPIC_API_KEY: "sk-ant-REPLACE-ME" diff --git a/examples/12-api-contract-validation/github-token-secret.yaml b/examples/12-api-contract-validation/github-token-secret.yaml new file mode 100644 index 00000000..1ffa06da --- /dev/null +++ b/examples/12-api-contract-validation/github-token-secret.yaml @@ -0,0 +1,9 @@ +apiVersion: v1 +kind: Secret +metadata: + name: github-token +type: Opaque +stringData: + # TODO: Replace with your GitHub Personal Access Token + # Required permissions: repo (for private repos), workflow (optional) + GITHUB_TOKEN: "ghp_REPLACE-ME" diff --git a/examples/12-api-contract-validation/pipeline.yaml b/examples/12-api-contract-validation/pipeline.yaml new file mode 100644 index 00000000..cc5bf19b --- /dev/null +++ b/examples/12-api-contract-validation/pipeline.yaml @@ -0,0 +1,50 @@ +# Stage 1: Identify all affected consumers +apiVersion: kelos.dev/v1alpha1 +kind: Task +metadata: + name: identify-consumers +spec: + type: claude-code + workspaceRef: + name: api-service-workspace + credentials: + type: api-key + secretRef: + name: anthropic-api-key + branch: api-migration/v2 + prompt: | + A breaking API change has been approved for the user-service API. + + 1. Read CONSUMERS.md or search the organization for repositories that import + this API's client package or reference its endpoints. + 2. For each consumer, identify which breaking fields/endpoints they use. + 3. Output a structured summary: consumer repo, affected endpoints, migration effort (low/medium/high). + + Commit a file `migration-plan.md` summarizing your findings and push the branch. +--- +# Stage 2: Generate migration guide (waits for identify-consumers to succeed) +apiVersion: kelos.dev/v1alpha1 +kind: Task +metadata: + name: generate-migration-guide +spec: + type: claude-code + workspaceRef: + name: api-service-workspace + credentials: + type: api-key + secretRef: + name: anthropic-api-key + branch: api-migration/v2 + dependsOn: + - identify-consumers + prompt: | + The consumer analysis is complete on branch + {{index .Deps "identify-consumers" "Results" "branch"}}. + + Read `migration-plan.md` and generate a comprehensive migration guide: + 1. Step-by-step instructions for each breaking change + 2. Before/after code examples for each consumer language (Go, Python, TypeScript) + 3. A deprecation timeline with recommended milestones + + Commit `MIGRATION_GUIDE.md` and open a PR with `gh pr create`. diff --git a/examples/12-api-contract-validation/taskspawner-cron.yaml b/examples/12-api-contract-validation/taskspawner-cron.yaml new file mode 100644 index 00000000..3f11b8ce --- /dev/null +++ b/examples/12-api-contract-validation/taskspawner-cron.yaml @@ -0,0 +1,40 @@ +apiVersion: kelos.dev/v1alpha1 +kind: TaskSpawner +metadata: + name: weekly-api-compatibility-audit +spec: + when: + cron: + schedule: "0 8 * * 1" # Every Monday at 8:00 AM UTC + taskTemplate: + type: claude-code + workspaceRef: + name: api-service-workspace + agentConfigRef: + name: api-validator-config + credentials: + type: api-key + secretRef: + name: anthropic-api-key + promptTemplate: | + Scheduled weekly API compatibility audit — {{.Time}} + + ## Instructions + + 1. Locate all API definition files in this repository (OpenAPI YAML/JSON, + protobuf .proto files, GraphQL .graphql schemas, or typed route definitions). + 2. Compare the current API surface against the latest released/tagged version: + - `git diff $(git describe --tags --abbrev=0)..HEAD -- '*.proto' '*.yaml' '*.json' '*.graphql'` + 3. Identify any unreleased breaking changes that have accumulated since the last release. + 4. Check for API hygiene issues: + - Deprecated fields without removal timelines + - Undocumented endpoints or parameters + - Inconsistent naming conventions across endpoints + - Missing or outdated API version headers + 5. Create a GitHub issue summarizing the audit findings with: + - A table of unreleased breaking changes + - Recommended actions (add deprecation notices, bump API version, etc.) + - Priority classification (critical / warning / info) + + Use `gh issue create` with the label `api-audit` for the findings report. + ttlSecondsAfterFinished: 3600 diff --git a/examples/12-api-contract-validation/taskspawner-webhook.yaml b/examples/12-api-contract-validation/taskspawner-webhook.yaml new file mode 100644 index 00000000..074845ee --- /dev/null +++ b/examples/12-api-contract-validation/taskspawner-webhook.yaml @@ -0,0 +1,54 @@ +apiVersion: kelos.dev/v1alpha1 +kind: TaskSpawner +metadata: + name: api-contract-validator +spec: + when: + githubWebhook: + events: + - "issue_comment" + filters: + - event: "issue_comment" + action: "created" + bodyContains: "/validate-api" + maxConcurrency: 2 + taskTemplate: + type: claude-code + workspaceRef: + name: api-service-workspace + agentConfigRef: + name: api-validator-config + credentials: + type: api-key + secretRef: + name: anthropic-api-key + branch: "kelos/api-validation-{{.ID}}" + promptTemplate: | + An API contract validation was requested. + + Event: {{.Event}} — {{.Action}} + Triggered by: @{{.Sender}} + PR URL: {{.URL}} + + ## Instructions + + 1. Identify all changes to API definition files in the current branch + (OpenAPI specs, protobuf definitions, GraphQL schemas, or typed API routes). + 2. Run structural diff tools if available (e.g., `oasdiff diff`, `buf breaking`). + 3. For each detected change, classify it: + - **Non-breaking**: additive fields, new optional parameters, new endpoints + - **Potentially breaking**: changed response shapes, renamed fields, type changes + - **Breaking**: removed fields/endpoints, changed required parameters + 4. For breaking changes, analyze whether any known consumer actually uses the + affected field/endpoint (check consumer repos if accessible). + 5. Post a review comment on the PR summarizing findings: + - List of changes with classification + - Impact assessment for each breaking change + - Suggested backward-compatible alternatives (deprecation, default values, versioning) + 6. If all changes are non-breaking, approve with a summary comment. + + Use `gh pr review` to post your findings. + metadata: + labels: + kelos.dev/use-case: "api-validation" + ttlSecondsAfterFinished: 3600 diff --git a/examples/12-api-contract-validation/workspace.yaml b/examples/12-api-contract-validation/workspace.yaml new file mode 100644 index 00000000..fcbaefb1 --- /dev/null +++ b/examples/12-api-contract-validation/workspace.yaml @@ -0,0 +1,10 @@ +apiVersion: kelos.dev/v1alpha1 +kind: Workspace +metadata: + name: api-service-workspace +spec: + # TODO: Replace with your API service repository URL + repo: https://github.com/your-org/your-api-service.git + ref: main + secretRef: + name: github-token diff --git a/examples/README.md b/examples/README.md index de5869e9..078b7782 100644 --- a/examples/README.md +++ b/examples/README.md @@ -22,6 +22,7 @@ Ready-to-use patterns and YAML manifests for orchestrating AI agents with Kelos. | [09-bedrock-credentials](09-bedrock-credentials/) | Run an agent using AWS Bedrock with static credentials or IRSA | | [10-taskspawner-github-webhook](10-taskspawner-github-webhook/) | Respond to GitHub webhook events (issues, PRs, pushes) in real time | | [11-taskspawner-linear-webhook](11-taskspawner-linear-webhook/) | Respond to Linear webhook events (issues, comments) in real time | +| [12-api-contract-validation](12-api-contract-validation/) | API contract validation pipeline with webhook, cron, and task dependencies | ## How to Use