R-API 3.59.0 shows test case passed although there are failed test steps

API ReadyAPI
, ,
1

Originally posted by sdeevers_starz on April 16, 2025

During our evaluation of R-API Release 3.59.0, we noticed if a test case has failed steps, the test case itself is marked as passed. This could lead to people thinking their tests are passing at a quick glimpse when in actuality, the test has failed.

Below is an example of a test with an SLA of 1000 milliseconds, which shows those failures in the “Transaction Log”, yet the test case shows the tests passed.

The associated “Test Case Options” have the “Fail Test Case on Error”, “Fail Test Case if it has failed Test Steps” selected, so something seems to not be working correctly.

998×510 111 KB

We are upgrading from R-API Release 3.53.0 to R-API Release 3.59.0 and we did not see this issue with R-API Release 3.53.0.

Has anyone else noticed this behavior with R-API Release 3.59.0?

2

Originally posted by phuong_pham on April 18, 2025

Yes, I’ve also encountered the same issue when running my tests using ReadyAPI version 3.59.0. Previously, I was using version 3.54, and this issue did not occur.

I found that setting Max Results to 0 in the Test Case Options seems to resolve the issue.
Hope SmartBear can look into this and provide a fix soon.

3

Originally posted by sdeevers_starz on April 23, 2025

Just a follow-up on the above noted issue with R-API 3.59.0.

In the current documentation for the “Default Test Case Options” the default value for “Max Results” is 5.

Making the change manually for each test case’s “Test Case Options”, “Max Value” to have a value of zero (0), would be an arduous task.

4

Originally posted by sdeevers_starz on April 29, 2025

Just checked the latest R-API Release, 3.60.0, and the issue of a test step failing and the Test Case and the Test Suite showing pass is still present :frowning:

This is core functionality that needs to be addressed as testers need to know when a test case or test suite has failed in the UI without having to drill into the “Transaction Log” after each run to see if something truly did fail.

5

Originally posted by sdeevers_starz on May 02, 2025

I have created a SB Support Ticket, 00707979, for the issue.

6

Originally posted by mnwill117 on May 09, 2025

We found that increasing the Max Results setting to be greater than or equal to the number of test steps in the test case accounted for all tests steps, and failed the test case if any failed.

7

Originally posted by joostdgfed on June 12, 2025

+1 on this. Fix on this from SmartBear is waiting too long.

Testcases also don’t show as failed in the generated jUnit reports. I’m not investing time on a workaround for this as this should be a quick hotfix.

8

Originally posted by joostdgfed on June 23, 2025

sdeevers_starz​ / Jimmie222​ / mnwill117​ / phuong_pham​ :

In new version 3.61.0 (release date 18/06/2025) there is a mention of bugfix :

  • An issue where a failed test would incorrectly display the wrong status (RIA-23395).

Hope that this relates to this issue. (Would be nice to seem some forum feedback here from SmartBear :-/ )

9

Originally posted by sdeevers_starz on June 23, 2025

I received notification this morning the above issue was addressed in R-API Release 3.61.0.

In the “Bug Fixes” section, you will see “RIA-23395”, the ticket associated to this issue.

Should anyone get this checked before me, please let us know if all is now well.

10

Originally posted by phuong_pham on June 24, 2025

I have checked version 3.61.0, and it appears to be working correctly. Test cases that were previously marked as ‘Pass’—despite some test steps failing—are now correctly marked as ‘Fail’

11

Originally posted by sam123 on June 25, 2025

Hi,

This issue has been successfully resolved in our latest release of ReadyAPI 3.61.0. Please ensure that you update to the latest version and test it out. Thanks!