Repository navigation
retry-total conflicts with html-formatter when exhaustion limit hit #1905
Description
Activity
cc/ @Enceradeira this is the first big issue we've seen come out of this introduction
- added 11 commits that reference this issue
on Sep 5, 2026 The cause:
willBeRetriedin theTestCaseFinishedmessage is computed from--retryalone, while the retry filter also honours--retry-total. Once the total is exhausted, the message claims a retry that never comes, and the HTML formatter hides the attempt.Reproducing it on
mainshowed two more defects in the same place:- cucumber-ruby passes the number of retries where the runner expects the number of attempts. With
--retry 1the flag is nevertrue, so the report shows a retried scenario twice. - The runner broadcasts
test_case_finishedbefore emitting the message. The retry filter re-runs the scenario from inside that event, so the first attempt's message comes out after the retry, with the retry's started id.
#1907 / cucumber/cucumber-ruby-core#344 fix the reported symptom but keep these two. I've opened an alternative pair that keeps the retry decision in one place and asserts the message values:
- Let a retry policy decide will_be_retried in the runner cucumber-ruby-core#345: the runner asks a retry policy, and emits the message before the event
- Report will_be_retried from the retry filter's decision #1910:
Cucumber::RetryPolicy, shared by the retry filter and the runner, plus a scenario checkingattempt,willBeRetriedand the started ids
Both are drafts until core 20 is released. Happy to adjust if you'd rather go another way.
- cucumber-ruby passes the number of retries where the runner expects the number of attempts. With
- added a commit that references this issue
on Oct 5, 2026 - added a commit that references this issue
on Oct 9, 2026
👓 What did you see?
The HTML report does not include scenarios which are skipped from being retried after exhaustion limit is hit
✅ What did you expect to see?
HTML formatter would show failures
📦 Which tool/library version are you using?
cucumber 11.1.1
html-formatter 23.1.0
🔬 How could we reproduce it?
Hit the exhaustion limit on a running context. So for example have a folder with 3 features and 3 scenarios. Have all 3 fail, set --retry 1 and --retry-total 2.
The third scenario won't feature in the report.
📚 Any additional context?
#1669 introduced it