LGatgfs
New Contributor
Joined 4 years ago
User Profile
User Widgets
Contributions
Re: Reporting - JUnit - Includes test cases being called by other test cases in test count
Hi mkrausem, I have been able to resolve my issue by ignoring the XML file that is generated for the test suite that houses testcase2. Our projects typically contain a separate test suite (named Framework) for functionality deemed common and suitable for modularisation. It means whenever we call the RunTestCase TestStep, it is always calling a test case in the Framework testsuite. Simply by excluding the Framework XML file that is generated following the completion of a test run on Azure DevOps we get a more realistic picture of the number of tests that have actually passed and failed. My rationale for the change back to the previous log behaviour: 1. We make use of the RunTestCase TestStep functionality quite a bit, meaning the number of testcases reported to have been executed in one test run is considerably more than the number of actual tests being executed, made even more pronounced by the fact that a large number of our test cases are data driven. 2. When you run a test case from another test case, the called test case is a subset of the parent test case. Yes, another test case is running but it is only a step in the parent test case and shouldn’t be treated as a separate test Yes, this workaround circumvents this change in functionality, but I feel the functionality isn’t quite right. I asked Smartbear what the rationale was for the original change but they have not responded. I hope this helps, Daniel.773Views1like0CommentsReporting - JUnit - Includes test cases being called by other test cases in test count
Hi all, It seems ReadyAPI reporting functionality has changed over the years. Previously, JUnit style reports only reported on tests that were explicitly initiated, whether at the project level, suite level or test case level. Now it seems any test case that is called from the original test case is also included in the count. Lets say for example we have two test cases, one, named testcase1, is a data driven test case that consists of a number of test steps including a step that calls the other test case. The other test case, testcase2, resides in a test suite that is disabled and is only ever called from testcase1. When I run testcase1 using testrunner.bat with the junit switches set (-j and -J), I expect to only see one result, testcase1 either passed or failed. However, the number of results returned includes the result for testcase1 + the results for testcase2 wherever it is called. Is there any way of reverting this behaviour as I preferred the original approach of reporting on tests that are explicitly initiated only? Regards, Daniel.961Views0likes5Comments