Forum Discussion

KMR's avatar
KMR
Occasional Contributor
11 years ago

Test step and assertion status using testRunner.results

I am trying the below code to get the status of the test step and assertions in the soapUi test case.
In the test case I am doing a loop so that the same request gets executed with multiple sets of data.

The issue I am facing is that for each iteration of the step result , assertion status and error message of the last instance of the step is returned.
Due to this I am not able to find out if the step passed in a particular iteration or not.

Is this an issue with soapUi ?
attached project has a very simplified version of what I am trying to implement)


for( r in testRunner.results )
{
 if ( r.testStep instanceof com.eviware.soapui.model.testsuite.Assertable)
 {
   log.info "TestStep [ " + r.testStep.name + " ] finished with status " + r.status
    for( assertion in r.testStep.assertionList )
    {
     log.info "Assertion [" + assertion.label + "] has status [" + assertion.status + "]"
     for( e in assertion.errors )
     log.info "-> Error [" + e.message + "]"     
    }
    //log.info  r.testStep.testRequest.response.contentAsString
 }
}


Log out put is below . For the first three lines I was expecting the assertion status to be Valid.
In the response that I got assertion passed, but it looks like the status from the last iteration of the step is being returned in the code.

Wed Sep 25 14:27:59 PDT 2013:INFO:TestStep [ GetInfoByZIP ] finished with status OK
Wed Sep 25 14:27:59 PDT 2013:INFO:Assertion [Citi Name Match] has status [FAILED] -- [Kalesh] This is not correct. For this instance the Assertion was a success.
Wed Sep 25 14:27:59 PDT 2013:INFO:-> Error [Missing token [Bellevue] in Response]

Wed Sep 25 14:27:59 PDT 2013:INFO:TestStep [ GetInfoByZIP ] finished with status FAILED
Wed Sep 25 14:27:59 PDT 2013:INFO:Assertion [Citi Name Match] has status [FAILED]
Wed Sep 25 14:27:59 PDT 2013:INFO:-> Error [Missing token [Bellevue] in Response]

I am using soapUi version 4.6.0 (Same is the behavious in 4.5.2 as well)
Note :
Instead of using testRunner.results , if I use testRunner.testCase.testSteps["stepname"] with in the loop , I am able to get the details correctly.
But I would prefer to use testRunner.results outside the loop if it gives me the correct output

3 Replies

Sort By
  • AndyYNWA's avatar
    AndyYNWA
    Occasional Contributor
    9 years ago
    Hi may I ask if you managed to sort this as I'mentioned struggling to to get the results of each step from my data driven suites
  • SmartBear_Suppo's avatar
    SmartBear_Suppo
    SmartBear Alumni (Retired)
    11 years ago
    I just ran your project, and I did not see it loop.

    Here's how it ran for me:
    Test started at 2013-09-26 16:09:01.772
    Step 1 [Start] OK: took 91 ms
    Step 2 [Properties] OK: took 0 ms
    Step 3 [GetInfoByZIP] OK: took 19339 ms
    Step 4 [Loop] OK: took 57 ms
    Step 5 [GetInfoByZIP] FAILED: took 669 ms
    -> [Citi Name Match] Missing token [Bellevue] in Response
    Step 6 [Loop] OK: took 3 ms
    Step 7 [Finish] OK: took 70 ms
    TestCase failed [Failing due to failed test step], time taken = 20229


    And the failure makes sense since you are looking for a string "Bellevue" in the response. But your response is:
    <soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
       <soap:Body>
          <GetInfoByZIPResponse xmlns="http://www.webserviceX.NET">
             <GetInfoByZIPResult>
                <NewDataSet xmlns="">
                   <Table>
                      <CITY>San Diego</CITY>
                      <STATE>CA</STATE>
                      <ZIP>92122</ZIP>
                      <AREA_CODE>619</AREA_CODE>
                      <TIME_ZONE>P</TIME_ZONE>
                   </Table>
                </NewDataSet>
             </GetInfoByZIPResult>
          </GetInfoByZIPResponse>
       </soap:Body>
    </soap:Envelope>


    Thanks,
    Michael Giller
    SmartBear Software
  • KMR's avatar
    KMR
    Occasional Contributor
    11 years ago
    Thanks Michael.
    Step response is correct..The issue that i am having is in getting the status in the Step Finish
    Can you please have a look at the script log, which I am printing from the step Finish.
    Its not matching with the real result.

    What I am trying to do is to Report the status at the end of Test case.

    (Regarding the loop : Step 3 & Step 5 are executing the same Test Step with a different input value.
    For the purpose of sharing , I just used a very simple script and a very crude way to set the data )