Forum Discussion

RBlack's avatar
RBlack
Occasional Contributor
7 years ago

failOnError versus failTestCaseOnErrors

SoapUI Pro 1.8.0 has these two TestCase Options which sound like they do the same thing:

 

Abort on Error: "Abort test if an error occurs"

Fail Test Case on Error: "Fail TestCase if it has failed TestSteps"

 

I assume the former Aborts the test case if some unexpected internal error occurs (which invalidates the test and for which no assertions would have been defined), whereas the latter Fails the test case if one of the test steps fails in a normal way (e.g. assertion failure).  Is that the case?

 

Related question: what is the difference between a test that Aborts and a test that Fails?  Is the distinction only detectable at the level of Groovy code inspecting the status of some object?  E.g. WsdlTestCaseRunner has methods isCanceled and isFailed but no mention of isAborted.

 

Please clarify.  Thanks.

3 Replies

    • RBlack's avatar
      RBlack
      Occasional Contributor

      Thanks for the link, though I find the explanation on that page a bit confusing:

      https://support.smartbear.com/readyapi/docs/soapui/ui/case.html

       

      Abort on ErrorStops the test run when a test step fails.
      Fail Test Case on ErrorMarks the test case as failed if a test step fails.
      Note:This option does not stop the test run.

       

      Although Abort on Error immediately stops execution of a specific test case upon an error, I know that SoapUI will continue executing subsequent test cases (i.e. that setting doesn't stop the overall project run).  

       

      In contrast, it seems "Fail Test Case on Error" just means that the test case will be marked as failed because a test step failed, but it keeps executing steps after the failed one.  

       

      So it seems that Fail refers to the final status of the test case but Abort refers to whether it continues executing.

       

      What would happen if Abort on Error was TRUE but Fail Test Case on Error was FALSE?  If a test step failed, what status would the test case have once it aborted?  

      • sanj's avatar
        sanj
        Super Contributor

        Honestly I think the second option that is mark test as failure makes no sense to me.

        If it fails just mark it as failed.

        if its set to False, I would not expect the test to be marked as Fail

        all its doing is creating confusion