Forum Discussion

sas's avatar
sas
Occasional Contributor
6 years ago

Data driven tests reporting

I have written data-driven tests (excel file data source) with one SOAPUI test case, This test case loops through each excel row (which is a test case) iteratively and executes the test steps. However, when I look at the report it shows one test case was executed with 'n' number of test steps. 

For example:

My SOAP-UI test has 4 test steps.

Test Step 1 - Groovy Script

Test Step 2 - POST

Test Step 3 - GET (This step has multiple assertions on Response) 

Test Step 4 - DELETE

 

Say, My excel data source has 3 test cases, say (testcase.xlsx) file looks like this.

TestCase                     Parm1       Parm2

This is Test Case 1      P1-A         P2-A

This is Test Case 2      P1-B         P2-B

This is Test Case 3      P1-C         P2-C

 

and as my SOAP-UI Test is looping and executing these tests... I want my report to look like this...

 

Test Case 1 

test step 1 was executed - Passed - in ##ms

test step 2 was executed - Passed - in ##ms

test step 3 was executed - Passed - in ##ms

                Assertion 1 - Passed

                Assertion 2 - Passed

test step 4 was executed - Passed - in ##ms

 

Test Case 2 

test step 1 was executed - Passed - in ##ms

test step 2 was executed - Passed - in ##ms

test step 3 was executed - Passed - in ##ms

                Assertion 1 - Passed

                Assertion 2 - Passed

test step 4 was executed - Passed - in ##ms

 

Test Case 3

test step 1 was executed - Passed - in ##ms

test step 2 was executed - Passed - in ##ms

test step 3 was executed - Passed - in ##ms

                Assertion 1 - Passed

                Assertion 2 - Passed

test step 4 was executed - Passed - in ##ms

 

How can I get report output for data-driven tests?

 

Your help is much appreciated. 

 

Thanks

Data-Driven Testing
Reporting
REST
  • nmrao's avatar
    nmrao
    Champion Level 3
    6 years ago
    From different posts, got your point.

    Default report generation is based on the SoapUI/ReadyAPI test case, but not based on the data set used in the test. And every knew this, I believe. However, many questions out there in the forum as users wanted the report differently.

    So, now the option is to see if it is feasible that way as mentioned in the question.
    Usually, when the test is executed, the tool generates some files for each test and I believe that those artifacts are used while generating the report. But, in case of data-driven, the same test is repeated as many times as the data available and it may stop once a row is failed or data set is complete. In the report, it can take only that as per my understanding.

    There is documentation available for custom reporting and also user can write some kind of plug-in if I am not mistaken.
    Hope you might have gone thru documentation. Otherwise take a look and see and try the methods suggested and come back with your observations.

    https://support.smartbear.com/readyapi/docs/testing/reports/managing.html
    https://support.smartbear.com/readyapi/docs/testing/reports/customizing.html
    https://support.smartbear.com/readyapi/docs/testing/reports/script.html
    https://support.smartbear.com/readyapi/docs/testing/reports/datasink/index.html

    It is quite possible that current reporting / customization might not exists the way it is required. Otherwise some one would have pointed. But still worth to look what is the nearest method to achieve it easily.

    Remember some plug-in to custom report, but not on top my head at the moment, try searching on the net.

    Just curious to know what is the current reporting that is being adopted? Junit, PDF, Allure or something else?
    • sonya_m's avatar
      sonya_m
      SmartBear Alumni (Retired)
      6 years ago

      Kudos for your reply nmrao! 

      Hi sas, thank you for creating a new topic for your issue and providing the details. Was nmrao’s reply helpful? 

      • sas's avatar
        sas
        Occasional Contributor
        6 years ago

        nmrao thanks for the suggestion, I am reviewing them and trying out ways to implement a solution.

        I will post to this group once I have a solution. 

         

        Specifically.... 

        I am trying out a solution using "datasink" to capture the test steps status while the test case loops through the data source and execute the test steps.  Datasink type subreport, which can later be used to generate the test case report. 

        The challenge is each row from datasource is a test case (datasource loop 1) with multiple test steps (data source loop 2), I am trying to capture the test results for each test step (loop 2)  and roll that status up to get the final status of the test case row (loop 1) in the datasource, which is not easily available. 

         

        I will try to see if I can create a simple project and upload it here so that the community can see the challenge and provide some input.

         

        Thanks

        sas