Difference between TestCaseRunner Summary and Reporting
When running tests via the testrunner we see differences between the TestCaseRunner Summary and the generated reporting.
In some cases the TestCaseRunner Summary fails the run, but the generated reporting does not give any errors (which is expected).
Is there something that can explain this behavior?
The project that created these results is very heavy on scripting and creating testcases on runtime based on input from an excel file. Its also very modular, so some testSuites are disabled, but testcases within those disabled testsuites are being called by other testcases.
Thanks for the reply.
What should I be looking at in the log? I know that there is an error being thrown in one of the groovy scripts. But if that triggers a failed Testcase in the testrunner summary, I would expect it would also trigger a failed testcase in de report. Even if we fix that error and we have a "Green" run, all counts (of testSuites, testcases, teststeps and assertions) are still way different in both reports. It seems like that the testrunner uses a different "source" than the report for the summary...