Skip to content

Fix re-quarantine criteria to only count post-unquarantine failures #66174

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
wtgodbe merged 2 commits into from
Apr 6, 2026
+4 −2
Merged

Fix re-quarantine criteria to only count post-unquarantine failures #66174

Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
459ec6c
Fix re-quarantine criteria to only count post-unquarantine failures
wtgodbe Apr 6, 2026
b689981
Update .github/workflows/test-quarantine.md
wtgodbe Apr 6, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions .github/workflows/test-quarantine.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -211,9 +211,11 @@ All of the following are true:

All of the following are true:
- The test was **recently unquarantined** (had its `[QuarantinedTest]` attribute removed within the past 14 days, detectable via `git log --since="14 days ago" -G 'QuarantinedTest' -- '*.cs'`)
- It has **at least one failure** in the observation window. Failing so soon after unquarantining means it was unquarantined too early. For these tests, search for the original closed quarantine issue (title prefix "Quarantine" referencing the test name) and **reopen** it rather than creating a new one in Step 2.4.
- It has **at least one failure that occurred AFTER the unquarantine commit was merged**. Use `git log` to find the merge date of the commit that removed the `[QuarantinedTest]` attribute, and only count failures from builds that started after that date. Failures from before the unquarantine do not count — they are from when the test was still quarantined. For these tests, search for the original closed quarantine issue (title prefix "Quarantine" referencing the test name) and **reopen** it rather than creating a new one in Step 2.4.
Comment thread
wtgodbe marked this conversation as resolved.
Outdated

Additionally, check for **class-level quarantine** candidates. If a **test class** has more than 3 total failures across multiple methods, you **must** investigate the error messages before deciding:
**Class-level quarantine (applies to both Case A and Case B)**

After identifying individual quarantine candidates from either case above, also check for **class-level quarantine** opportunities. If a **test class** has more than 3 total failures across multiple methods, you **must** investigate the error messages before deciding:

1. For each failure in the class, extract the error message and stack trace from the Helix console log. When searching the console log for `[FAIL]`, also capture the lines immediately following it — these contain the `Error Message:` and `Stack Trace:` sections.
2. Compare the error messages and stack traces across all failing methods in the class. Look for the same exception type, similar call chains, or a shared root cause.
Expand Down
Loading