Data driven tests reporting
I have written data-driven tests (excel file data source) with one SOAPUI test case, This test case loops through each excel row (which is a test case) iteratively and executes the test steps. Howeve...
Forum Discussion
From different posts, got your point.
Default report generation is based on the SoapUI/ReadyAPI test case, but not based on the data set used in the test. And every knew this, I believe. However, many questions out there in the forum as users wanted the report differently.
So, now the option is to see if it is feasible that way as mentioned in the question.
Usually, when the test is executed, the tool generates some files for each test and I believe that those artifacts are used while generating the report. But, in case of data-driven, the same test is repeated as many times as the data available and it may stop once a row is failed or data set is complete. In the report, it can take only that as per my understanding.
There is documentation available for custom reporting and also user can write some kind of plug-in if I am not mistaken.
Hope you might have gone thru documentation. Otherwise take a look and see and try the methods suggested and come back with your observations.
https://support.smartbear.com/readyapi/docs/testing/reports/managing.html
https://support.smartbear.com/readyapi/docs/testing/reports/customizing.html
https://support.smartbear.com/readyapi/docs/testing/reports/script.html
https://support.smartbear.com/readyapi/docs/testing/reports/datasink/index.html
It is quite possible that current reporting / customization might not exists the way it is required. Otherwise some one would have pointed. But still worth to look what is the nearest method to achieve it easily.
Remember some plug-in to custom report, but not on top my head at the moment, try searching on the net.
Just curious to know what is the current reporting that is being adopted? Junit, PDF, Allure or something else?
Default report generation is based on the SoapUI/ReadyAPI test case, but not based on the data set used in the test. And every knew this, I believe. However, many questions out there in the forum as users wanted the report differently.
So, now the option is to see if it is feasible that way as mentioned in the question.
Usually, when the test is executed, the tool generates some files for each test and I believe that those artifacts are used while generating the report. But, in case of data-driven, the same test is repeated as many times as the data available and it may stop once a row is failed or data set is complete. In the report, it can take only that as per my understanding.
There is documentation available for custom reporting and also user can write some kind of plug-in if I am not mistaken.
Hope you might have gone thru documentation. Otherwise take a look and see and try the methods suggested and come back with your observations.
https://support.smartbear.com/readyapi/docs/testing/reports/managing.html
https://support.smartbear.com/readyapi/docs/testing/reports/customizing.html
https://support.smartbear.com/readyapi/docs/testing/reports/script.html
https://support.smartbear.com/readyapi/docs/testing/reports/datasink/index.html
It is quite possible that current reporting / customization might not exists the way it is required. Otherwise some one would have pointed. But still worth to look what is the nearest method to achieve it easily.
Remember some plug-in to custom report, but not on top my head at the moment, try searching on the net.
Just curious to know what is the current reporting that is being adopted? Junit, PDF, Allure or something else?
Kudos for your reply nmrao!
Hi sas, thank you for creating a new topic for your issue and providing the details. Was nmrao’s reply helpful?
nmrao thanks for the suggestion, I am reviewing them and trying out ways to implement a solution.
I will post to this group once I have a solution.
Specifically....
I am trying out a solution using "datasink" to capture the test steps status while the test case loops through the data source and execute the test steps. Datasink type subreport, which can later be used to generate the test case report.
The challenge is each row from datasource is a test case (datasource loop 1) with multiple test steps (data source loop 2), I am trying to capture the test results for each test step (loop 2) and roll that status up to get the final status of the test case row (loop 1) in the datasource, which is not easily available.
I will try to see if I can create a simple project and upload it here so that the community can see the challenge and provide some input.
Thanks
sas
