Groovy Rerun Failed Test Case Execution Report
I am using groovy script to rerun failed test cases but in Junit-Style Report , the test cases are still coming as failed even though they got passed after several retries.
Could you please let me know how to ensure that only the final execution test status is available in the report.
Please find the link for groovy script used for rerunning failed test cases : https://community.smartbear.com/t5/API-Functional-Security-Testing/Groovy-script-to-re-run-failed-tests/td-p/168404
The point here is not the execution but the reporting, after multiple retries if the Test Cases is passed it is marked as Pass at Test Suit level on the tool interface but in the Test Suit Printable Report is still shows fail.
It is being addressed the issue incorrectly. First look for the reason why it is happening. If the services works with definitive behavior, then there is no issue of reporting.
I have added assertion as a Test step in the test case. Since, the assertion can pass only if the value in response matches the assertion value, the test case may fail on multiple retries but it will keep on re running for a fixed number of times.
Please check with your team why such indefinitve behavior?
How will the end user / customer use it if one can't predict the expected response based on the input?
For each fixed input, the response should always be predictable. And respective assertions must always pass. Otherwise, I am afraid the application behavior can be acceptable.