Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
125 changes: 125 additions & 0 deletions examples/12-api-contract-validation/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,125 @@
# 12 — API Contract Validation

An end-to-end API contract validation pipeline that uses PR-triggered
validation, scheduled compatibility audits, and a multi-step consumer
update pipeline to catch and resolve breaking API changes.

## Use Case

Detect breaking changes in API definitions (OpenAPI, protobuf, GraphQL)
when PRs are opened, run weekly audits for unreleased drift, and
coordinate migration across consumer repositories when intentional
breaking changes are approved.

AI agents add value beyond static diff tools by reasoning about semantic
intent, analyzing real consumer impact, proposing backward-compatible
alternatives, and generating targeted migration guides.

## Resources

| File | Kind | Purpose |
|------|------|---------|
| `credentials-secret.yaml` | Secret | Anthropic API key for the agent |
| `github-token-secret.yaml` | Secret | GitHub token for cloning and PR creation |
| `workspace.yaml` | Workspace | Git repository containing API definitions |
| `agentconfig.yaml` | AgentConfig | Shared instructions for the API validator agent |
| `taskspawner-webhook.yaml` | TaskSpawner | PR-triggered validation via `/validate-api` comment |
| `taskspawner-cron.yaml` | TaskSpawner | Weekly cross-service compatibility audit |
| `pipeline.yaml` | Task (x2) | Consumer impact analysis and migration guide pipeline |

## Configurations

### 1. PR-Triggered API Validation (Webhook)

Listens for `/validate-api` comments on PRs. The agent checks out the PR
branch, runs structural diff tools if available, classifies each change,
and posts a review summarizing findings with backward-compatible
alternatives for any breaking changes.

### 2. Scheduled Compatibility Audit (Cron)

Runs every Monday at 8:00 AM UTC. The agent compares the current API
surface against the latest tagged release, identifies unreleased breaking
changes and API hygiene issues, and creates a GitHub issue with findings.

### 3. Consumer Update Pipeline (Task Dependencies)

A two-stage pipeline for coordinating migration after an intentional
breaking change is approved:

```
identify-consumers (Task)
│ analyzes which consumers use the affected endpoints
│ outputs: migration-plan.md
generate-migration-guide (Task, dependsOn: [identify-consumers])
│ reads migration-plan.md
│ generates step-by-step migration guide with code examples
│ opens a PR with the guide
```

## Steps

1. **Edit the secrets** — replace placeholders in `credentials-secret.yaml`
and `github-token-secret.yaml`.

2. **Edit `workspace.yaml`** — set your API service repository URL.

3. **Review `agentconfig.yaml`** — customize the agent instructions for your
API format and conventions.

4. **Deploy the webhook-triggered validator:**

```bash
kubectl apply -f credentials-secret.yaml -f github-token-secret.yaml \
-f workspace.yaml -f agentconfig.yaml -f taskspawner-webhook.yaml
```

5. **Deploy the scheduled audit (optional):**

```bash
kubectl apply -f taskspawner-cron.yaml
```

6. **Run the consumer update pipeline (when needed):**

```bash
kubectl apply -f pipeline.yaml
```

7. **Trigger a validation** — comment `/validate-api` on any PR in the
configured repository.

8. **Watch spawned Tasks:**

```bash
kubectl get tasks -w
```

9. **Cleanup:**

```bash
kubectl delete -f examples/12-api-contract-validation/
```

## Complementary Issues

This use case works with current Kelos primitives but would benefit from
proposed API extensions:

| Issue | Enhancement | Benefit |
|-------|-------------|---------|
| #778 | `filePatterns` filter | Auto-trigger on API file changes instead of manual `/validate-api` |
| #884 | Cross-repo propagation | Create migration PRs directly in consumer repos |
| #842 | `readOnlyWorkspaces` | Read consumer repos without full write access |
| #881 | `contextSources` | Inject previous audit results into validation prompts |

## Notes

- The webhook TaskSpawner uses `issue_comment` with `bodyContains` as a
workaround until `filePatterns` filtering (#778) is available.
- The pipeline Tasks share the same `branch` value, which serializes them
automatically in addition to the explicit `dependsOn` ordering.
- Adjust `maxConcurrency` on the webhook TaskSpawner based on your expected
PR volume.
25 changes: 25 additions & 0 deletions examples/12-api-contract-validation/agentconfig.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
apiVersion: kelos.dev/v1alpha1
kind: AgentConfig
metadata:
name: api-validator-config
spec:
agentsMD: |
# API Contract Validator Agent

You are an API contract validation specialist. Your role is to analyze API
definition changes for backward compatibility and consumer impact.

## Key Conventions
- Always classify changes as non-breaking, potentially-breaking, or breaking
- A "breaking change" means any change that could cause existing consumers
to fail without code modifications
- When suggesting alternatives, prefer deprecation over removal
- Reference the specific API versioning strategy used in this project
- Include concrete code examples in migration guides

## Tools Available
- Use `git diff` to compare API definitions between versions
- Use `gh api` to query for dependent repositories or PRs
- Use `gh pr review` to post validation results
- If `oasdiff`, `buf`, or `graphql-inspector` are available, use them first
for structural analysis before adding semantic evaluation
8 changes: 8 additions & 0 deletions examples/12-api-contract-validation/credentials-secret.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
apiVersion: v1
kind: Secret
metadata:
name: anthropic-api-key
type: Opaque
stringData:
# TODO: Replace with your Anthropic API key
ANTHROPIC_API_KEY: "sk-ant-REPLACE-ME"
9 changes: 9 additions & 0 deletions examples/12-api-contract-validation/github-token-secret.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
apiVersion: v1
kind: Secret
metadata:
name: github-token
type: Opaque
stringData:
# TODO: Replace with your GitHub Personal Access Token
# Required permissions: repo (for private repos), workflow (optional)
GITHUB_TOKEN: "ghp_REPLACE-ME"
50 changes: 50 additions & 0 deletions examples/12-api-contract-validation/pipeline.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
# Stage 1: Identify all affected consumers
apiVersion: kelos.dev/v1alpha1
kind: Task
metadata:
name: identify-consumers
spec:
type: claude-code
workspaceRef:
name: api-service-workspace
credentials:
type: api-key
secretRef:
name: anthropic-api-key
branch: api-migration/v2
prompt: |
A breaking API change has been approved for the user-service API.

1. Read CONSUMERS.md or search the organization for repositories that import
this API's client package or reference its endpoints.
2. For each consumer, identify which breaking fields/endpoints they use.
3. Output a structured summary: consumer repo, affected endpoints, migration effort (low/medium/high).

Commit a file `migration-plan.md` summarizing your findings and push the branch.
---
# Stage 2: Generate migration guide (waits for identify-consumers to succeed)
apiVersion: kelos.dev/v1alpha1
kind: Task
metadata:
name: generate-migration-guide
spec:
type: claude-code
workspaceRef:
name: api-service-workspace
credentials:
type: api-key
secretRef:
name: anthropic-api-key
branch: api-migration/v2
dependsOn:
- identify-consumers
prompt: |
The consumer analysis is complete on branch
{{index .Deps "identify-consumers" "Results" "branch"}}.

Read `migration-plan.md` and generate a comprehensive migration guide:
1. Step-by-step instructions for each breaking change
2. Before/after code examples for each consumer language (Go, Python, TypeScript)
3. A deprecation timeline with recommended milestones

Commit `MIGRATION_GUIDE.md` and open a PR with `gh pr create`.
40 changes: 40 additions & 0 deletions examples/12-api-contract-validation/taskspawner-cron.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
apiVersion: kelos.dev/v1alpha1
kind: TaskSpawner
metadata:
name: weekly-api-compatibility-audit
spec:
when:
cron:
schedule: "0 8 * * 1" # Every Monday at 8:00 AM UTC
taskTemplate:
type: claude-code
workspaceRef:
name: api-service-workspace
agentConfigRef:
name: api-validator-config
credentials:
type: api-key
secretRef:
name: anthropic-api-key
promptTemplate: |
Scheduled weekly API compatibility audit — {{.Time}}

## Instructions

1. Locate all API definition files in this repository (OpenAPI YAML/JSON,
protobuf .proto files, GraphQL .graphql schemas, or typed route definitions).
2. Compare the current API surface against the latest released/tagged version:
- `git diff $(git describe --tags --abbrev=0)..HEAD -- '*.proto' '*.yaml' '*.json' '*.graphql'`
3. Identify any unreleased breaking changes that have accumulated since the last release.
4. Check for API hygiene issues:
- Deprecated fields without removal timelines
- Undocumented endpoints or parameters
- Inconsistent naming conventions across endpoints
- Missing or outdated API version headers
5. Create a GitHub issue summarizing the audit findings with:
- A table of unreleased breaking changes
- Recommended actions (add deprecation notices, bump API version, etc.)
- Priority classification (critical / warning / info)

Use `gh issue create` with the label `api-audit` for the findings report.
ttlSecondsAfterFinished: 3600
54 changes: 54 additions & 0 deletions examples/12-api-contract-validation/taskspawner-webhook.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
apiVersion: kelos.dev/v1alpha1
kind: TaskSpawner
metadata:
name: api-contract-validator
spec:
when:
githubWebhook:
events:
- "issue_comment"
filters:
- event: "issue_comment"
action: "created"
bodyContains: "/validate-api"
maxConcurrency: 2
taskTemplate:
type: claude-code
workspaceRef:
name: api-service-workspace
agentConfigRef:
name: api-validator-config
credentials:
type: api-key
secretRef:
name: anthropic-api-key
branch: "kelos/api-validation-{{.ID}}"
promptTemplate: |
An API contract validation was requested.

Event: {{.Event}} — {{.Action}}
Triggered by: @{{.Sender}}
PR URL: {{.URL}}

## Instructions

1. Identify all changes to API definition files in the current branch
(OpenAPI specs, protobuf definitions, GraphQL schemas, or typed API routes).
2. Run structural diff tools if available (e.g., `oasdiff diff`, `buf breaking`).
3. For each detected change, classify it:
- **Non-breaking**: additive fields, new optional parameters, new endpoints
- **Potentially breaking**: changed response shapes, renamed fields, type changes
- **Breaking**: removed fields/endpoints, changed required parameters
4. For breaking changes, analyze whether any known consumer actually uses the
affected field/endpoint (check consumer repos if accessible).
5. Post a review comment on the PR summarizing findings:
- List of changes with classification
- Impact assessment for each breaking change
- Suggested backward-compatible alternatives (deprecation, default values, versioning)
6. If all changes are non-breaking, approve with a summary comment.

Use `gh pr review` to post your findings.
metadata:
labels:
kelos.dev/use-case: "api-validation"
ttlSecondsAfterFinished: 3600
10 changes: 10 additions & 0 deletions examples/12-api-contract-validation/workspace.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
apiVersion: kelos.dev/v1alpha1
kind: Workspace
metadata:
name: api-service-workspace
spec:
# TODO: Replace with your API service repository URL
repo: https://github.com/your-org/your-api-service.git
ref: main
secretRef:
name: github-token
1 change: 1 addition & 0 deletions examples/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ Ready-to-use patterns and YAML manifests for orchestrating AI agents with Kelos.
| [09-bedrock-credentials](09-bedrock-credentials/) | Run an agent using AWS Bedrock with static credentials or IRSA |
| [10-taskspawner-github-webhook](10-taskspawner-github-webhook/) | Respond to GitHub webhook events (issues, PRs, pushes) in real time |
| [11-taskspawner-linear-webhook](11-taskspawner-linear-webhook/) | Respond to Linear webhook events (issues, comments) in real time |
| [12-api-contract-validation](12-api-contract-validation/) | API contract validation pipeline with webhook, cron, and task dependencies |

## How to Use

Expand Down
Loading