Commit f676570
committed
fix(ci): make the review cap per-PR and require exhaustive findings [skip changelog]
Two problems with what #1290 shipped, both surfaced by that PR taking 13
review rounds.
The 3-run cap was per-commit, not per-PR. The API query filtered on
`head_sha`, so the counter reset on every push - a PR could take unlimited
reviews at three per commit, which is the opposite of a cap. When the reviewer
pointed this out I corrected the log message and the skip comment to say "for
this commit", making the wording match the bug instead of fixing it. #1290 is
the evidence: 15 successful review runs on one PR under a "3 run" cap.
Now scoped with `branch=<head ref>`, which spans the PR's whole life, with
`--paginate` and a summing awk so a PR with more than 100 runs still counts
correctly. Verified against #1290's branch: returns 15, and an unknown branch
returns 0 rather than empty.
The prompt also never asked for completeness, which is why 13 rounds happened.
Five of them were one class of bug - advice contradicting its own check -
surfacing a corner at a time: crate direction, root direction, mixed case,
per-file lines, all-misnamed. Each round found one instance, I fixed that
instance, and the next round found the next. Under a real per-PR cap that
pattern exhausts the budget without converging.
The prompt now requires sweeping the whole class before reporting, with the
specific generalizations that would have collapsed those five rounds into one:
check parallel lists for the same omission, grep for other places repeating a
contradicted claim, ask what the opposite direction does, check sibling files.
Also asks for confidence and verification method per finding, and to say
plainly when something could not be verified rather than presenting inference
as measurement - the reviewer did this well unprompted and it is worth making
explicit.1 parent b976987 commit f676570
1 file changed
Lines changed: 46 additions & 14 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
56 | 56 | | |
57 | 57 | | |
58 | 58 | | |
| 59 | + | |
59 | 60 | | |
60 | 61 | | |
61 | 62 | | |
62 | 63 | | |
63 | 64 | | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
64 | 70 | | |
65 | 71 | | |
66 | 72 | | |
67 | | - | |
68 | | - | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
69 | 77 | | |
70 | 78 | | |
71 | 79 | | |
| |||
75 | 83 | | |
76 | 84 | | |
77 | 85 | | |
78 | | - | |
79 | | - | |
80 | | - | |
81 | | - | |
82 | | - | |
83 | | - | |
84 | | - | |
85 | | - | |
86 | | - | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
87 | 92 | | |
88 | 93 | | |
89 | 94 | | |
90 | | - | |
| 95 | + | |
91 | 96 | | |
92 | 97 | | |
93 | | - | |
| 98 | + | |
94 | 99 | | |
95 | 100 | | |
96 | 101 | | |
| |||
190 | 195 | | |
191 | 196 | | |
192 | 197 | | |
| 198 | + | |
| 199 | + | |
| 200 | + | |
| 201 | + | |
| 202 | + | |
| 203 | + | |
| 204 | + | |
| 205 | + | |
| 206 | + | |
| 207 | + | |
| 208 | + | |
| 209 | + | |
| 210 | + | |
| 211 | + | |
| 212 | + | |
| 213 | + | |
| 214 | + | |
| 215 | + | |
| 216 | + | |
| 217 | + | |
| 218 | + | |
| 219 | + | |
| 220 | + | |
| 221 | + | |
| 222 | + | |
| 223 | + | |
| 224 | + | |
193 | 225 | | |
194 | 226 | | |
195 | 227 | | |
| |||
199 | 231 | | |
200 | 232 | | |
201 | 233 | | |
202 | | - | |
| 234 | + | |
0 commit comments