Reporting - JUnit - Includes test cases being called by other test cases in test count

Hi mkrausem,

I have been able to resolve my issue by ignoring the XML file that is generated for the test suite that houses testcase2. Our projects typically contain a separate test suite (named Framework) for functionality deemed common and suitable for modularisation. It means whenever we call the RunTestCase TestStep, it is always calling a test case in the Framework testsuite. Simply by excluding the Framework XML file that is generated following the completion of a test run on Azure DevOps we get a more realistic picture of the number of tests that have actually passed and failed.

My rationale for the change back to the previous log behaviour:

1. We make use of the RunTestCase TestStep functionality quite a bit, meaning the number of testcases reported to have been executed in one test run is considerably more than the number of actual tests being executed, made even more pronounced by the fact that a large number of our test cases are data driven.

2. When you run a test case from another test case, the called test case is a subset of the parent test case. Yes, another test case is running but it is only a step in the parent test case and shouldn’t be treated as a separate test

Yes, this workaround circumvents this change in functionality, but I feel the functionality isn’t quite right.

I asked Smartbear what the rationale was for the original change but they have not responded.

I hope this helps,

Daniel.