Skip to content

Add reusable AI code review workflow#3

Merged
mahangu merged 12 commits intotrunkfrom
add/ai-review-workflow
Apr 16, 2026
Merged

Add reusable AI code review workflow#3
mahangu merged 12 commits intotrunkfrom
add/ai-review-workflow

Conversation

@mahangu
Copy link
Copy Markdown
Member

@mahangu mahangu commented Apr 16, 2026

Changes proposed in this Pull Request

Adds a reusable AI code review workflow for WooCommerce repos, powered by Claude (Opus 4.6) via anthropics/claude-code-action.

Part of the AI Code Reviews for wpcom/MC/Woo project. See QAO-392.

What it does

  • Reusable workflow (workflow_call) that extension repos call with a ~15 line caller file
  • Gated to members of the happiness-engineers and qualityops GitHub teams
  • Reviews PRs for backwards compatibility, security, WooCommerce best practices, PHP 8.x compat, and performance
  • Posts inline comments on specific lines via the batched review API (single notification)
  • Supports incremental follow-up reviews on subsequent pushes (two-tier: clean incremental diff when possible, prompt-based dedup after rebases)
  • Discovers and applies per-repo ai-review-rules.md files (read from base branch for security)
  • Concurrency control: cancels stale reviews when new commits are pushed
  • Fork PRs blocked, draft PRs skipped
  • Requires AI_CODE_REVIEW_ANTHROPIC_API_KEY org-level secret

Security review

  • No secrets exposed in public YAML
  • Shell injection mitigated (all GitHub expressions via env vars, not inline)
  • Tool access restricted: Write limited to ai_review_payload.json, gh api limited to pulls/reviews and pulls/comments endpoints
  • ai-review-rules.md read from base branch (not PR branch) to prevent prompt injection
  • Previous review detection filters to github-actions[bot] authored reviews only
  • id-token: write required by claude-code-action for OIDC authentication
  • Risk assessment: LOW (see PR discussion)

Caller workflow example

name: AI Code Review
on:
  pull_request:
    types: [opened, synchronize, ready_for_review, reopened]

concurrency:
  group: ai-review-${{ github.event.pull_request.number }}
  cancel-in-progress: true

jobs:
  review:
    uses: woocommerce/.github/.github/workflows/ai-code-review.yml@main
    secrets:
      AI_CODE_REVIEW_ANTHROPIC_API_KEY: ${{ secrets.AI_CODE_REVIEW_ANTHROPIC_API_KEY }}

How to test the changes in this Pull Request

  1. Merge this PR to trunk
  2. In a target repo that has the AI_CODE_REVIEW_ANTHROPIC_API_KEY secret, merge the caller workflow above to the default branch
  3. Open a non-draft PR from a user on the happiness-engineers or qualityops team
  4. Verify the AI review posts inline comments as a single batched review
  5. Push a follow-up commit and verify incremental review (Tier 1: Resolved/Still Outstanding/New)

Testing that has already taken place

Workflow reviewed by 5 automated agents: action compatibility, prompt quality, shell correctness, security, and bash security. All critical findings fixed. OIDC workflow validation requires merge to default branch before end-to-end testing.

Reusable workflow for WooCommerce extension repos using
anthropics/claude-code-action with Claude Opus 4.6.

Gated to @woocommerce/happiness-engineers team members.
Reviews PRs for backwards compatibility, security, WooCommerce
best practices, PHP 8.x compat, and performance.

Supports incremental follow-up reviews on subsequent pushes.

Linear: QAO-392

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@mahangu mahangu self-assigned this Apr 16, 2026
@mahangu mahangu marked this pull request as draft April 16, 2026 03:05
Mahangu Weerasinghe and others added 2 commits April 16, 2026 08:35
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Adds pull_request trigger so the workflow runs on this repo's own PRs.
Hardcodes mahangu as allowed user (bypasses team check).
Adds fallback defaults for inputs that are empty on pull_request trigger.
Revert this commit before merging to main.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@mahangu mahangu marked this pull request as ready for review April 16, 2026 03:21
Mahangu Weerasinghe and others added 9 commits April 16, 2026 08:52
The action uses OIDC for GitHub token setup and fails without this permission.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Removes pull_request trigger and hardcoded user check.
Restores team membership check via GitHub API.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Falls back to team API check for all other users.
Revert before merging to main.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
When GH_TOKEN_OVERRIDE secret is provided, skips the Anthropic OIDC
workflow validation that requires the workflow to exist on the default
branch. This allows testing on PR branches.

Remove GH_TOKEN_OVERRIDE from callers once workflow is merged to trunk.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Uses GH_TOKEN_OVERRIDE_MAHANGU to bypass OIDC workflow validation.
Revert this commit before merging to main.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Removes pull_request trigger, hardcoded users, and PAT override.
Workflow is now production-ready for merge to trunk.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Checks happiness-engineers and qualityops teams by default.
Loops through comma-separated team slugs, allows if member of any.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Cancels in-progress review if a new push arrives on the same PR.
Prevents unnecessary API costs from stacked runs.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Replace team membership check (GITHUB_TOKEN can't read org teams)
  with hardcoded allowed_users list (mahangu, anant1811 for testing)
- Restrict Read tool to /home/runner/work/** to prevent reading
  /proc/self/environ (secret exfiltration risk)
- Quote all RUNNER_TEMP paths

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@mahangu mahangu merged commit 8b0716e into trunk Apr 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant