Junit style report not being accurate in Data-Driven tests

We use TestRunner (be able to hook up to build system) to execute the test suite and generate JUnit style reports, currently, SoapUI pro doesn't treat each dataSource row as a test case, therefore, if one out of ten dataSource row (test case) fails, it fails the whole test cases, we want this report to accurately show how many really fails and how many really passes in the report Summary page.

Please see the screenshows, one is the Summary page, another one is Drill-Down Faulures page.

I have talked to Jeshtha about this issue.

At Oracle, we have many licenses in our devision alone, the report not being accurate is a BIG issue for the management.

1 Comment

kbw
Contributor
7 years ago
Agreed with this. And it's not only for Junit style reports, it's for any form or reporting, including the transaction log, or the new Dashboards. They all report that 1 test case ran.

This would preferably be an option either the test case or on the datasource step itself. I lean toward the datasource, since you might have multiple datasources in the test case and one of them is the main driver of what constitutes a real test case.

FYI, others have complained about this too:

https://community.smartbear.com/t5/SoapUI-Pro/Ready-API-junit-reports-to-show-each-datasource-row-as-testcase/td-p/129542

Junit style report not being accurate in Data-Driven tests

1 Comment

Related Content

Data-Driven Loop in BDD Tests

Data Driven Loops

Data driven to put fluid data

Ready API data driven testing

Using Excel Sheet for Data Driven Loop with Empty Cells