Reporting - JUnit - Includes test cases being called by other test cases in test count

Question

Hi all,&nbsp;It seems ReadyAPI reporting functionality has changed over the years. Previously, JUnit style reports only reported on tests that were explicitly initiated, whether at the project level, suite level or test case level. Now it seems any test case that is called from the original test case is also included in the count.&nbsp;Lets say for example we have two test cases, one, named testcase1, is a data driven test case that consists of a number of test steps including a step that calls the other test case. The other test case, testcase2,&nbsp; resides in a test suite that is disabled and is only ever called from testcase1. When I run testcase1 using testrunner.bat with the junit switches set (-j and -J), I expect to only see one result, testcase1 either passed or failed. However, the number of results returned includes the result for testcase1 + the results for testcase2 wherever it is called.&nbsp;Is there any way of reverting this behaviour as I preferred the original approach of reporting on tests that are explicitly initiated only?&nbsp;Regards,Daniel.

lgatgfs · Answer

Cheers nmrao. Ticket raised.

mkrausem · Answer

Hi,&nbsp;I raised the same issues before.This function changed from 3.1. to 3.2.&nbsp;The support answered me that there is no option to change this and I should change my reporting. (Case #00451382).&nbsp;Did you get a solution for changing this?&nbsp;Kind Regards, Michael

lgatgfs · Answer

Hi&nbsp;mkrausem,&nbsp;I have been able to resolve my issue by ignoring the XML file that is generated for the test suite that houses testcase2. Our projects typically contain a separate test suite (named Framework) for functionality deemed common and suitable for modularisation. It means whenever we call the RunTestCase TestStep, it is always calling a test case in the Framework testsuite. Simply by excluding the Framework XML file that is generated following the completion of a test run on Azure DevOps we get a more realistic picture of the number of tests that have actually passed and failed.&nbsp;My rationale for the change back to the previous log behaviour:&nbsp;1. We make use of the RunTestCase TestStep functionality quite a bit, meaning the number of testcases reported to have been executed in one test run is considerably more than the number of actual tests being executed, made even more pronounced by the fact that a large number of our test cases are data driven.2. When you run a test case from another test case, the called test case is a subset of the parent test case. Yes, another test case is running but it is only a step in the parent test case and shouldn’t be treated as a separate test&nbsp;Yes, this workaround circumvents this change in functionality, but I feel the functionality isn’t quite right.&nbsp;I asked Smartbear what the rationale was for the original change but they have not responded.&nbsp;I hope this helps,Daniel.

lgatgfs · Answer

BTW, I have ReadyAPI 3.6.0

nmrao · Answer

You may open a support ticket for the same.

Forum Discussion

Reporting - JUnit - Includes test cases being called by other test cases in test count

5 Replies

Related Content

[TechCorner Challenge #9] A script to include custom details to the report

Calling

Test Execution Report to include # of Completed Steps/Total?

PDF / WORD Export (including ZS-Report graphs) from Confluence

Call Stack

Recent Discussions

Parameterising JDBC Connection Settings..Can you do it?

Test Runner - Report All in One File

GroovyScript Help - Repeating JSON Attributes Have Specific Value.....some of the time?

JDBC results are stored in the ReadyAPI project

MyFirstPlugin - button Project menu in Ready Api