Forum Discussion

fwseb's avatar
fwseb
New Contributor
4 years ago

More correct assertion of testcase

I use a groovy script to cancel (or abort) the test execution (if some precondition is satisfied):

 

// test step "SendSellOrderVerifyRequest" is a setup/precondition of the testcase.

def rspHeader = context.testCase.testSteps["SendSellOrderVerifyRequest"].testRequest.response.responseHeaders
if (! rspHeader["#status#"].contains("HTTP/1.1 200 OK")) {
    context.testRunner.cancel("TestCase cancelled, because ...... ")
}

 

This gives the test case "Cancelled" result in ReadyAPI, as well as in the test log:

2021-03-23T13:51:45.4138758Z 14:51:45,399 INFO  [SoapUIProTestCaseRunner] Finished running TestCase [xxxxxx], time taken: 1259ms, status: CANCELED
 
But, in Azure pipeline, this test case is counted as "Passed", probably because "Canceled" is not a supported outcome of test result in Azure:
> There are 15 possible outcomes for a test result: Aborted, Blocked, Error, Failed, Inconclusive, In progress, None, Not applicable, Not executed, Not impacted, Passed, Paused, Timeout, Unspecified, and Warning.
 
My question is:
How many test result outcomes are ReadyAPI supporting, besides Passed, Failed and Canceled? is there any way to set the test assertion to "Aborted" or "Inconclusive" etc that Azure supports, to align with Azure? If not can you add the feature of having more assertion outcomes such as "Aborted"?
 

3 Replies

  • nmrao's avatar
    nmrao
    Champion Level 3

    fwseb wrote:

     

    This gives the test case "Cancelled" result in ReadyAPI, as well as in the test log:

    2021-03-23T13:51:45.4138758Z 14:51:45,399 INFO  [SoapUIProTestCaseRunner] Finished running TestCase [xxxxxx], time taken: 1259ms, status: CANCELED

    Does this work when the same project is called / execute from command-line using testrunner utility?

     

    NOTE: No-voice in Azure pipeline.

     

    By the way, what command do you use in Azure pipeline to execute the project?

    • fwseb's avatar
      fwseb
      New Contributor

      I haven't tried to run the test project by command line myself, but the command line that is used in pipeline is:

      C:\WINDOWS\system32\cmd.exe /D /S /C ""C:\Program Files\Maven\apache-maven-3.5.2\bin\mvn.cmd" -f <my test project pom>.xml test --fail-at-end"
       
      • nmrao's avatar
        nmrao
        Champion Level 3
        Thank you for that.
        Do you see the same behavior if you run the command locally to execute the project?