Skip to content

Auto-comment on PR template CI failure #10015

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open

Auto-comment on PR template CI failure #10015

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
702fc38
Auto comment mistakes
raman976 Jul 27, 2025
27825a6
auto error detection
raman976 Jul 27, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
59 changes: 39 additions & 20 deletions .github/workflows/validate.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -49,38 +49,57 @@ jobs:

validate-pr-template:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We might need to set a concurrency restriction on this, so that you don't end up in a race if the user edits the body multiple times in quick succession, with the wrong comment being the one that ends up sent to GitHub?

if: github.event_name == 'pull_request'

runs-on: ubuntu-latest

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, and you're going to need to add a permissions stanza that is specific to this job with increased permissions, so that the bot can write comments on the PR.

steps:
- name: Validate PR template
- name: Validate PR template and comment
uses: actions/github-script@v7
with:
script: |
const body = context.payload.pull_request.body || "";
const pr = context.payload.pull_request;
const body = pr.body || "";

// 1. Check both checkboxes
const checkbox1 = /\[x\].*?There is reasonable content/i.test(body);
const checkbox2 = /\[x\].*?I have read and accepted/i.test(body);

// 2. URL must be on the exact "The site content can be seen at ..." line
// https://regex101.com/r/N36fsT
const urlLineRegex = /^[ \t]*-[ \t]*The site content can be seen at[ \t]+(?:<)?(?:\[.*?\]\()?https?:\/\/[^\s>()]+(?:\))?(?:>)?/m;
const urlMatch = urlLineRegex.exec(body);
const urlValid = urlMatch !== null;

// 3. Explanation must follow the blockquote marker
const urlValid = urlLineRegex.exec(body) !== null;
const explanationMatch = />\s*The site content is(?:\s*|\s*\n)(.+)/i.exec(body);
const explanation = explanationMatch && explanationMatch[1].trim().length > 10;

if (!checkbox1 || !checkbox2 || !urlValid || !explanation) {
core.setFailed(
"❌ PR template is not properly filled:\n" +
`Checkbox1: ${checkbox1 ? '✅' : '❌'}\n` +
`Checkbox2: ${checkbox2 ? '✅' : '❌'}\n` +
`URL on correct line: ${urlValid ? '✅' : '❌'}\n` +
`Explanation: ${explanation ? '✅' : '❌'}`
);
const failed = !checkbox1 || !checkbox2 || !urlValid || !explanation;

if (!failed) {
console.log("✅ PR template is valid.");
return;
}

const commentBody = `❌ **PR template is not properly filled:**\n\n- Checkbox1 (reasonable content): ${checkbox1 ? "✅" : "❌"}\n- Checkbox2 (read and accepted): ${checkbox2 ? "✅" : "❌"}\n- URL on correct line: ${urlValid ? "✅" : "❌"}\n- Explanation (> The site content is...): ${explanation ? "✅" : "❌"}\n\nPlease update your PR description accordingly.`;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we might want some more explicit guidance here on exactly the markup that the check is expecting to find when a check has failed? I also wonder if we simply don't need to include the bits that're okay, to reduce the information that a requester needs to read through?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we use an actual multi-line template literal here, instead of encoded \n chars? Should make it much easier to work with and understand what the resulting comment body will be?


const { data: comments } = await github.rest.issues.listComments({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: pr.number,
});

const existingComment = comments.find(c =>
c.user.login === 'github-actions[bot]' &&
c.body.includes("**PR template is not properly filled:**")
);

if (existingComment) {
await github.rest.issues.updateComment({
owner: context.repo.owner,
repo: context.repo.repo,
comment_id: existingComment.id,
body: commentBody
});
} else {
console.log("✅ PR template format is valid.");
await github.rest.issues.createComment({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: pr.number,
body: commentBody
});
}

core.setFailed("PR template validation failed.");
Loading