Feature Request: Add Support for Indicating Soft Assertion Failures in RED Color for Specific Cucumber Steps

[sharangithub14]

1. Description:

Currently, Allure does not provide a way to mark a specific Cucumber step as failed (indicated in RED) when using soft assertions. While soft assertions allow execution to continue after a failure, the parent Cucumber step is always marked as passed unless an exception is thrown. This behavior makes it challenging to highlight particular failures in the report for debugging and analysis.

Request a feature that allows users to explicitly mark the status of a specific Cucumber step as failed (RED color) even if no exception is thrown. This would improve visibility of soft assertion failures in Allure reports.

2. How We Used Earlier:

We developed a custom function to verify soft assertion failures and used it to mark the Cucumber step as failed while also attaching screenshots and custom error messages to the Allure report.

The following step was used to mark the Cucumber step as failed:
allureReporter.default.endStep('failed');

This approach worked seamlessly in version @wdio/[email protected], where we could use endStep('failed') to indicate the parent Cucumber step's failure status directly.

3. Issue Observed in Latest Version:

After upgrading to @wdio/[email protected], we encountered the following issue:

[0-0] 2025-01-17T13:22:31.765Z DEBUG @wdio/utils:shim: Finished to run "afterStep" hook in 1ms
[0-0] 2025-01-17T13:22:31.791Z DEBUG @wdio/utils:shim: Finished to run "after" hook in 0ms
[0-0] 2025-01-17T13:22:31.793Z ERROR @wdio/runner: Error: There are no active steppable entities!

Upon investigation, we found that Allure has enforced a strict step lifecycle in version 8, requiring every step to begin with startStep and conclude with endStep. While this ensures lifecycle integrity, the use of startStep creates a hierarchy in the Allure report, introducing sub-steps under the parent step. This behavior is not desirable for us, as it adds unnecessary nesting and deviates from the flat structure we require for individual steps in the report.

Due to this error unable to use endStep('failed') as it stops the execution and we also attempted to use addStep, but it does not meet our requirements because it creates additional entries in the report rather than modifying the status of the parent Cucumber step.

4. Challenge Faced Due to Issue:

1. Soft assertion failures are not visually prominent in Allure reports, as the parent step is marked as passed.
2. Report analysis becomes difficult in large test suites where partial failures are critical.
3. If reusable step is used in multiple places within end-to-end scenarios, it's difficult to debug where exactly it's failing
4. Earlier in quick glance able identify on which cucumber step exactly it's failing, while now it's more challenging

5. Request for New Function/Feature:

We propose adding a new feature to the Allure API that allows users to:

1. Dynamically mark a specific Cucumber step as failed (RED color) without throwing an error and ensuring execution continuity.
2. Log soft assertion failures directly to the parent step instead of creating additional steps or attachments.
3. Automatically mark the overall scenario as failed if any soft assertion fails within the steps.

Ex: Let say if soft assert failed then will call new feature and pass parameter as failed "allureReporter.default.markStepAsFailed('failed')". then the specific cucumber step mark as RED where the new feature is called.

6. Advantage of New Function/Feature:

1. Improved Reporting: The parent Cucumber step will accurately reflect soft assertion failures, enhancing report readability.
2. Better Debugging: Users can easily identify specific failures in the context of the original test step without additional clutter.
3. The challenges mentioned above can be effectively addressed by leveraging the new feature.

Adding this feature would enhance Allure's functionality, especially for teams using soft assertions and BDD frameworks like Cucumber. It aligns with the goal of making Allure reports more actionable, readable, and efficient for debugging complex test scenarios.

Kindly let us know if you require any further details or clarifications. Additionally, please inform us if there are any alternative solutions available.

[Yashwanthk-personal]

This feature could be a good addition for report analysis. it will reduce time, Please do consider this as enhancement.