Skip to content

Add pr-lifecycle contributor skill - #1346

Merged
Alexandre Zollinger Chohfi (azchohfi) merged 10 commits into
mainfrom
azchohfi-pr-lifecycle-skill
Oct 7, 2026
Merged

Alexandre Zollinger Chohfi (azchohfi) merged 10 commits into
mainfrom
azchohfi-pr-lifecycle-skill

Conversation

@azchohfi

@azchohfi Alexandre Zollinger Chohfi (azchohfi) commented Oct 6, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Adds a contributor-only .github/skills/pr-lifecycle skill. It takes a PR to a clean, current-head ready-for-review state, so we no longer have to ask each session to run the review loop by hand. It is modeled on the WinApp CLI pr-lifecycle skill (microsoft/winappCli#896, microsoft/winappCli#915), but uses a different review strategy:

  1. Prepare: snapshot the live PR, keep unrelated work intact, integrate the base, and set agent-preparing.

  2. One pr-review pass: run pr-review once and fix Critical/High/Medium findings (Low only if trivial), with tests and validation.

  3. Present: write the PR body from the repo template.

  4. Copilot review loop with no round cap: request an @copilot review on the head. The skill waits for it using a session wakeup instead of polling. Every round then requires:

    • green CI on the head: the required checks from the live rulesets and branch protection (matched on check name and app), the always-on Detect changes job, and no failing check run
    • a reply on every Copilot and github-code-quality thread (fixed with the SHA, or disputed with evidence), then a resolve
    • a reply on every human thread and every actionable human review body or comment; human threads are left open for that reviewer to resolve

    It re-requests a review only after pushing a real fix, which keeps the loop from spinning on disputed items. It exits on a 🟢 Approval recommended verdict, or when no meaningful findings remain.

  5. Finish: set ready-for-review / agent-blocked, re-check every signal, clear the wakeup, and report. It never merges or enables auto-merge.

Safety rules: review comments, CI logs and bot reports are treated as untrusted input, not instructions. PR code only runs locally when the branch is in this repo or belongs to the person running the skill.

Also adds the skill to .github/skills/README.md and to the Repo skills list in AGENTS.md. The three lifecycle labels (agent-preparing, agent-blocked, ready-for-review) now exist in the repo.

Linked issue / spec

N/A. This is contributor workflow tooling.

Test plan

Contributor Markdown only; no product code changes.

  • Frontmatter (name, description, infer), relative links and in-page anchors resolve, and code fences are balanced (local script): passed
  • git diff --cached --check: passed
  • Checked each command the skill depends on against live data:
    • gh pr edit --add-reviewer "@copilot" (gh 2.101)
    • confirming the Copilot review request through the timeline and the head's check run
    • check runs (with app id) and commit statuses for the head; ruleset and classic-protection required checks
    • the paginated GraphQL reviewThreads query (authors show as copilot-pull-request-reviewer / github-code-quality)
    • Copilot verdict parsing from 40 recent PRs (🟢 / 🟡 / 🔵 / quota failure)
  • Ran the skill end to end on this PR: one pr-review pass, then Copilot review rounds until a review came back with no findings. Every round had green CI, and every thread got a reply and was resolved.

Risk / breaking changes

None for shipped packages: .github/skills/ is not packed into the agent kit. Running this skill does push commits and resolve threads on the PR it's pointed at, and only when someone explicitly asks for that.

Run pr-review once and fix Critical/High/Medium findings, then loop GitHub
Copilot review rounds until no meaningful findings remain. Every round
requires green CI and a reply + resolve on each Copilot and
github-code-quality thread. Tracks readiness with agent-preparing /
agent-blocked / ready-for-review labels and waits via a session wakeup.
Never merges.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 4551b177-1353-46d8-ba5b-347bc58fedda
@azchohfi Alexandre Zollinger Chohfi (azchohfi) added the agent-preparing Agent is addressing feedback or completing required validation and CI label Oct 6, 2026
Leave human review threads for the reviewer to resolve, treat PR content
as untrusted input, read the live required-check policy, page nested
thread comments, and pass reply text to gh api as raw strings.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 4551b177-1353-46d8-ba5b-347bc58fedda

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

The workflow can miss human feedback and required checks, mishandle fork PRs, and apply readiness against a stale base.

Review effort: Balanced
Findings: 5 Medium severity

Open (5)
What changed in this PR

Adds a contributor-only skill for preparing PRs, addressing findings, monitoring CI, and coordinating review rounds.

Changes:

  • Defines the five-phase pr-lifecycle workflow.
  • Documents lifecycle labels and contributor usage.
  • Adds the skill to repository guidance.
File Description
.github/​skills/​pr-lifecycle/​SKILL.md Defines the lifecycle workflow and commands.
.github/​skills/​README.md Registers and describes the skill.
AGENTS.md Adds the skill to contributor guidance.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread .github/skills/pr-lifecycle/SKILL.md Outdated
Comment thread .github/skills/pr-lifecycle/SKILL.md Outdated
Comment thread .github/skills/pr-lifecycle/SKILL.md Outdated
Comment thread .github/skills/pr-lifecycle/SKILL.md Outdated
Comment thread .github/skills/pr-lifecycle/SKILL.md Outdated
Re-check the base SHA as well as the head after labeling, commit only
when there are fixes, read required checks from classic branch protection
as well as rulesets, handle body-only reviews and top-level comments,
keep --repo on the PR's base repository, and confirm the Copilot request
via the timeline (Copilot is absent from reviewRequests).

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 4551b177-1353-46d8-ba5b-347bc58fedda

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Local execution of untrusted PR code and two GitHub-state validation gaps must be addressed.

Review effort: Balanced
Findings: 1 High severity

Open (1)
Resolved since last review (5)

Comment thread .github/skills/pr-lifecycle/SKILL.md
Only build or test PR code locally when the head is a branch in the base
repository or belongs to the user running the skill; other fork PRs rely
on CI or an isolated environment. Skip the agent's own reply-reviews when
reading review bodies.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 4551b177-1353-46d8-ba5b-347bc58fedda

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

The no-PR path and fork/request snapshot commands contain operational defects that can block or misdirect the lifecycle.

Review effort: Balanced
Findings: None

Resolved since last review (1)
Previously missed (3)

In code that hasn't changed since last review

Medium severity Apply agent-preparing label only after PR creation

.github/​skills/​pr-lifecycle/​SKILL.md:93

This sequence also covers the “No PR yet” branch, but it tries to apply agent-preparing before the PR is created. gh pr edit has no target at that point, so lifecycle startup fails for the creation path. Defer the label until immediately after creation, while retaining the existing-PR path here.

Medium severity Include head repository details in the PR snapshot

.github/​skills/​pr-lifecycle/​SKILL.md:328

Phase 1 requires snapshotting and fetching the actual head repository, but this only returns the head ref name/SHA. For a fork PR, $repo deliberately remains the base repository, and the author is not guaranteed to own the head repository, so the agent cannot reliably identify the remote it must fetch or push. Include the supported head-repository fields in the snapshot.

Medium severity Match Copilot request events to the current head commit

.github/​skills/​pr-lifecycle/​SKILL.md:332

This query accepts any historical Copilot request on the PR. After the first round, a failed/no-op request for a new head still appears registered because the old event matches, leaving the lifecycle waiting for a review that was never queued. Filter the timeline event by its commit_id matching $headSha (or require an event newer than this request).

Label a newly created PR only after it exists, snapshot the head
repository for fork fetch/push, and confirm the Copilot request against
the current head (check run, or a timeline event newer than the request)
instead of any historical event.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 4551b177-1353-46d8-ba5b-347bc58fedda

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

The CI gate can accept a required check from the wrong GitHub App because it compares policy entries by name only.

Review effort: Balanced
Findings: 1 High severity

Open (1)

Comment thread .github/skills/pr-lifecycle/SKILL.md Outdated
Match required checks on the policy's full identity (context plus pinned
app_id / integration_id) and look at commit statuses as well as check
runs.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 4551b177-1353-46d8-ba5b-347bc58fedda

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

The round-completion gate can race with review comments published after their associated checks finish.

Review effort: Balanced
Findings: None

Resolved since last review (1)
Previously missed (1)

In code that hasn't changed since last review

Medium severity Wait for current-head review findings before declaring completion

.github/​skills/​pr-lifecycle/​SKILL.md:181

A completed check is not sufficient to declare the round complete: the next line notes that github-code-quality publishes inline comments after Analyze (csharp) finishes, so a wakeup in that interval can read the old thread set and reach the exit conditions before the new findings appear. Require the current-head Copilot review to be visible, then perform a bounded post-check settle/re-read of reviews, threads, and comments before evaluating the gates.

Require the current-head Copilot review itself (not only its check run)
and a short settle-then-re-read before treating a round as complete, so
comments posted after their check finishes aren't missed.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 4551b177-1353-46d8-ba5b-347bc58fedda

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

Resume, review filtering, and status pagination gaps can produce incomplete or blocked lifecycle runs.

Review effort: Balanced
Findings: None

Previously missed (3)

In code that hasn't changed since last review

Medium severity Resume at Phase 3 after checkpointing completion

.github/​skills/​pr-lifecycle/​SKILL.md:118

A resumed run can bypass Phase 3 entirely. The checkpoint records only pr-review: done@<sha>, so an interruption after writing that marker but before refreshing the PR body will resume at Phase 4 and may finish with a stale or template-incomplete body. Resume at Phase 3 instead; that phase is already written to preserve concurrent human edits and is safe to repeat.

Medium severity Exclude only empty COMMENTED reviews from the current account

.github/​skills/​pr-lifecycle/​SKILL.md:247

This skips substantive reviews authored by the account running the lifecycle, not just the empty review wrappers created by thread replies. For example, a maintainer who previously submitted a body-only CHANGES_REQUESTED review and later runs this skill would have that feedback ignored while the PR can satisfy the exit gate. Filter only empty COMMENTED reviews from the current account.

Medium severity Paginate combined status contexts before evaluating CI

.github/​skills/​pr-lifecycle/​SKILL.md:362

The combined-status endpoint is paginated, but this command reads only its default first page (30 status contexts). If a required legacy context falls on a later page, the CI gate treats it as missing and can keep an otherwise green PR in agent-preparing. Paginate this read just like check runs.

Resume at Phase 3 rather than Phase 4 so the PR body is always refreshed,
skip only empty COMMENTED reviews from the running account, and paginate
the combined commit status read.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 4551b177-1353-46d8-ba5b-347bc58fedda

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

The lifecycle can accept stale checks after a base update or report green CI when no CI job ran.

Review effort: Balanced
Findings: None

Previously missed (2)

In code that hasn't changed since last review

Medium severity Invalidate stale checks after a base-only update

.github/​skills/​pr-lifecycle/​SKILL.md:76

A base-only move does not invalidate any of the checks this skill reads: Phase 4 keys both the Copilot review and CI results to headRefOid. Returning to the loop with the same head therefore reuses checks produced before the new base and can immediately restore ready-for-review without validating the new merged state. Require the new base to be integrated (or otherwise obtain fresh checks for the new merge result) before resuming the gate.

Medium severity Require current-head CI results before marking the gate green

.github/​skills/​pr-lifecycle/​SKILL.md:219

This gate is vacuously green when no CI jobs have started. That occurs for approval-gated fork workflows—the exact case where the trust rule says to rely on CI—and the Copilot check alone can satisfy the current “every existing check” test. This repo has an unconditional pull-request CI / Detect changes job (.github/workflows/ci.yml:3-4,19-22), so require that current-head result (or explicitly block on an absent/awaiting-approval run) before calling CI green.

Treat a base-only move as stale checks (integrate the new base to get a
fresh head, CI and review), and require the always-on CI 'Detect
changes' job on the head so the CI gate can't pass with no CI at all;
fork runs awaiting approval are agent-blocked.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 4551b177-1353-46d8-ba5b-347bc58fedda

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

Required-check outcome handling and the final label-write race can incorrectly block or stale the lifecycle state.

Review effort: Balanced
Findings: None

Previously missed (2)

In code that hasn't changed since last review

Medium severity Revalidate all readiness signals after writing the ready label

.github/​skills/​pr-lifecycle/​SKILL.md:74

The post-write verification only re-reads labels and the two SHAs. A check can become pending/failing or a review comment/thread can arrive between Phase 5's final snapshot and the label write without changing either SHA, leaving ready-for-review on a PR that no longer meets the stated exit conditions. Re-read all readiness signals after the write and restore agent-preparing if any changed.

Medium severity Accept skipped and neutral conclusions for required check runs

.github/​skills/​pr-lifecycle/​SKILL.md:211

This gate rejects required check runs whose conclusion is skipped or neutral, even though GitHub treats both as passing required-status-check outcomes. A path-filtered required job can therefore leave the lifecycle stuck despite branch protection being satisfied. Accept success/skipped/neutral for check runs while continuing to require success for legacy commit statuses.

Accept skipped/neutral for required check runs (success only for commit
statuses), and re-read every readiness signal after writing
ready-for-review, not just the head/base SHAs.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 4551b177-1353-46d8-ba5b-347bc58fedda

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

The skill performs authenticated pushes, PR mutations, and asynchronous readiness decisions that warrant final human validation.

Review effort: Balanced
Findings: None

@azchohfi Alexandre Zollinger Chohfi (azchohfi) added ready-for-review Agent work and technical checks complete; awaiting review, not approval or merge and removed agent-preparing Agent is addressing feedback or completing required validation and CI labels Oct 7, 2026
@azchohfi
Alexandre Zollinger Chohfi (azchohfi) merged commit 60f65b2 into main Oct 7, 2026
22 checks passed
@azchohfi
Alexandre Zollinger Chohfi (azchohfi) deleted the azchohfi-pr-lifecycle-skill branch October 7, 2026 01:13
@azchohfi Alexandre Zollinger Chohfi (azchohfi) removed the ready-for-review Agent work and technical checks complete; awaiting review, not approval or merge label Oct 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants