Ask a Question

Difference between TestCaseRunner Summary and Reporting

JochemNS
New Contributor

Difference between TestCaseRunner Summary and Reporting

Hello,

 

When running tests via the testrunner we see differences between the TestCaseRunner Summary and the generated reporting.

 

In some cases the TestCaseRunner Summary fails the run, but the generated reporting does not give any errors (which is expected).

 

Is there something that can explain this behavior?

 

The project that created these results is very heavy on scripting and creating testcases on runtime based on input from an excel file. Its also very modular, so some testSuites are disabled, but testcases within those disabled testsuites are being called by other testcases. 

 

An example:

JochemNS_1-1660224835946.png

 

JochemNS_3-1660225010339.png

 

 

3 REPLIES 3
nmrao
Community Hero

That's strange. Please have look in the logs for more information why it is showing like that.


Regards,
Rao.
JochemNS
New Contributor

Thanks for the reply.

 

What should I be looking at in the log? I know that there is an error being thrown in one of the groovy scripts. But if that triggers a failed Testcase in the testrunner summary, I would expect it would also trigger a failed testcase in de report. Even if we fix that error and we have a "Green" run, all counts (of testSuites, testcases, teststeps and assertions) are still way different in both reports. It seems like that the testrunner uses a different "source" than the report for the summary... 

nmrao
Community Hero

I said in the context why two different results; I mean basically both should match.



Regards,
Rao.
cancel
Showing results for 
Search instead for 
Did you mean: