research: find a least-privilege additional AI PR reviewer

[absolutepraya]

Goal

Find an additional AI-assisted pull request reviewer that can complement the existing CodeRabbit setup without weakening the repository's review-only safety model.

The target is:

free + useful + review-only + least privilege

CodeRabbit remains the current accepted baseline reviewer.

Why this issue changed

This issue originally tracked a rollout of Qodo, Sourcery, and Graphite alongside CodeRabbit. Live research and installation exposed constraints that make all three unsuitable for the intended setup:

Candidate	Result	Reason
Qodo	Rejected	Permanent zero-cost access requires open-source qualification/application, which does not fit the desired low-friction free-plan model.
Sourcery	Rejected	The installed GitHub App exposed repository contents write plus Actions/workflows write permissions, exceeding the approved least-privilege ceiling. Its first PR review also produced one intentional/optional suggestion and one clear factual false positive.
Graphite Agent	Rejected	The GitHub App permission model includes read/write repository contents and Actions/workflows access as part of the broader Graphite product, exceeding the approved review-only ceiling.

The durable evaluation history and safety policy are being captured in PR #41.

Hard requirements for future candidates

Cost

• zero paid subscription cost for the intended repository usage;
• no silent paid upgrade if limits or pricing change;
• avoid recurring application/qualification friction unless explicitly accepted as a new trade-off.

GitHub scope and permissions

• install only for absolutepraya/karakeep;
• repository contents must be read-only;
• pull request/review/comment write access is acceptable when needed to publish feedback;
• issues/comments and checks/statuses may be read/write only where needed for review interaction;
• reject candidates requiring:
  • repository contents write;
  • Actions/workflows write;
  • repository administration;
  • secrets/environments access;
  • equivalent broad code-mutation capabilities.

A vendor statement that review mode will not use a broad permission does not override this requirement. Capability matters as well as intended behavior.

Review behavior

• review/comments only;
• no automatic commits;
• no automatic pushes;
• no automatic application of suggestions;
• no autonomous coding/fixing agents;
• no reviewer-driven branch mutation;
• no AI reviewer as a required merge gate;
• deterministic GitHub Actions remain authoritative.

Quality

A candidate must be smoke-tested on a real non-draft PR before adoption.

Evaluate findings as:

1. confirmed defect;
2. valid but optional;
3. intentional behavior;
4. false positive;
5. ambiguous or behavior-changing.

Reviewer comments are evidence to verify, not instructions to change intended behavior. Agreement between multiple AI reviewers increases investigation priority but does not make a finding automatically correct.

Research workflow

For each candidate:

• Verify current official pricing/free-tier terms.
• Confirm whether free access requires an application or qualification process.
• Inspect the exact GitHub App permission model before treating the integration as accepted.
• Confirm the App can be scoped only to absolutepraya/karakeep.
• Reject excessive write/admin/secrets permissions.
• Enable review-only behavior.
• Smoke-test on a real non-draft PR.
• Confirm it does not commit, push, apply fixes, or mutate the branch.
• Confirm no reviewer status becomes a required merge gate.
• Inspect review output for useful unique findings, false positives, generic advice, and duplicate noise.
• Record the result here and in docs/ai-code-review.md before adoption.

Acceptance criteria

• At least one additional reviewer is found that meets all cost, permission, behavior, and quality requirements.
• The accepted integration is documented in docs/ai-code-review.md.
• CodeRabbit and deterministic CI behavior are not regressed.
• Rejected candidates remain documented so future maintainers do not repeat the same evaluation without a material product change.

Related work

PR #41 records the original reviewer-army design, the failed Qodo/Sourcery/Graphite evaluation, and the resulting least-privilege policy.