Correct test Case execution gives a 'Failed' status

[krogold]

Hello,

I have a strange problem with testCase execution.

I have a test case with the following structure:

Setup script:

  set session_id = 0

  set session_index = 0

  store custom properties in an external file

Test Case:

  data source

    SOAP - authentication on components (gives session_id for each component)

    groovy - recover useful data (store session_id in custom prop session_id_<session_index>)

  source loop

  SOAP - start asynchronous service on component 1

  SOAP - check status on component 1

  goto 'check status' while status = Pending

  SOAP - check status on component 2

  goto 'disconnect terminals' is session_id = 0

  SOAP disconnect

  groovy disconnect terminal : loop for each session_index: set session_id = session_id_<session_index> and use it in the SOAP disconnect step (I use run test step)

TearDown Script : restore the custom properties

All of these steps execute with success, I don't have any error logged but the overall test has a FAILED status.

What is wrong ? Is there something to configure to get more logging ?

thanks for any help

EDIT : Is it possible that the test gets failed because of my conditional gotos ? as long as my check status request is not valid I loop on it and continue when it no longer fails ... but before reaching the 'success' status it failed at least once

Alexandre

[Nastya_Khovrina]

Hi Alexandre,

Thank you for your email. From the Running a Test Case article: 

The Status of the test case will be set to FAILED if one of the following conditions is met:

• The test case was aborted due to an internal error.

• The test case was aborted because of a failed test step, or if there were failed test steps and the Fail Test Case on Error option in the TestCase Options dialog is selected.

• A script in any of the triggered events, scripts, or handlers failed the test case with testRunner.failexplicitly.

You can find the final state of the test case in the Run Log.

So, please check all these conditions.

[krogold]

Indeed, I chose to set the "fail testCase if it has failed TestSteps" in order to detect when a test is failing but, actually, at the end of the test, none of my teststeps is "failed" and I don't have any Error log information.

[Nastya_Khovrina]

Alexandre,

It's difficult to find out the root cause of the issue without a project file. I recommend that you open a new case with our Support team at: https://support.smartbear.com/message/?prod=ReadyAPI and provide your project file.

[krogold]

ok, I'll do that

thank you

[krogold · Accepted solution]

Finally I found a workaround for this problem:

I checked the results and found out that some of my test steps where considered as failed in testRunner.results. I set the following bit of code in my teardown script tab

for (testStep in testRunner.getResults()){
	log.info "status " + testStep.getTestStep().getName() + " : " + testStep.getStatus()
}

These steps were the ones on which I was looping, waiting for the status different from 'pending'. At the end of the test, as the status was finally 'success' the step was set as OK in the testCase (green-flagged), even if several occurrences of the testStep were 'failed'

So I found I could update those status doing the following for the concerned steps:

for (results in testRunner.results){
 // selection condition
    results.status = "OK"
}

I just have to make sure I apply this only on the concerned testSteps.

But at the end the overall status is still FAILED, still in my teardown script:

log.info "TEST RUNNER STATUS after update" + testRunner.getStatus()

If I'm sure of my steps and result, I can overwrite it as follows:

testRunner.status = "FINISHED"
log.info "TEST RUNNER STATUS after update" + testRunner.getStatus

And my overall test is ok (green bar)

I know it's a bit tricky, but as long as I set the proper conditions to update those parameters, I'm sure not to hide real failures.