Forum Discussion

krogold's avatar
krogold
Regular Contributor
8 years ago

Correct test Case execution gives a 'Failed' status

Hello,

 

I have a strange problem with testCase execution.

I have a test case with the following structure:

Setup script:

  set session_id = 0

  set session_index = 0

  store custom properties in an external file

Test Case:

  data source

    SOAP - authentication on components (gives session_id for each component)

    groovy - recover useful data (store session_id in custom prop session_id_<session_index>)

  source loop

  SOAP - start asynchronous service on component 1

  SOAP - check status on component 1

  goto 'check status' while status = Pending

  SOAP - check status on component 2

  goto 'disconnect terminals' is session_id = 0

  SOAP disconnect

  groovy disconnect terminal : loop for each session_index: set session_id = session_id_<session_index> and use it in the SOAP disconnect step (I use run test step)

 

TearDown Script : restore the custom properties

 

All of these steps execute with success, I don't have any error logged but the overall test has a FAILED status.

What is wrong ? Is there something to configure to get more logging ?

 

thanks for any help

 

EDIT : Is it possible that the test gets failed because of my conditional gotos ? as long as my check status request is not valid I loop on it and continue when it no longer fails ... but before reaching the 'success' status it failed at least once

 

 

Alexandre

 

  • krogold's avatar
    krogold
    8 years ago

    Finally I found a workaround for this problem:

    I checked the results and found out that some of my test steps where considered as failed in testRunner.results. I set the following bit of code in my teardown script tab

     

    for (testStep in testRunner.getResults()){
	log.info "status " + testStep.getTestStep().getName() + " : " + testStep.getStatus()
}

    These steps were the ones on which I was looping, waiting for the status different from 'pending'. At the end of the test, as the status was finally 'success' the step was set as OK in the testCase (green-flagged), even if several occurrences of the testStep were 'failed'

    So I found I could update those status doing the following for the concerned steps:

    for (results in testRunner.results){
     // selection condition
        results.status = "OK"
    }

    I just have to make sure I apply this only on the concerned testSteps.

     

    But at the end the overall status is still FAILED, still in my teardown script:

    log.info "TEST RUNNER STATUS after update" + testRunner.getStatus()

    If I'm sure of my steps and result, I can overwrite it as follows:

    testRunner.status = "FINISHED"
log.info "TEST RUNNER STATUS after update" + testRunner.getStatus

    And my overall test is ok (green bar)

    I know it's a bit tricky, but as long as I set the proper conditions to update those parameters, I'm sure not to hide real failures.

     

     

  • Nastya_Khovrina's avatar
    Nastya_Khovrina
    SmartBear Alumni (Retired)
    8 years ago

    Hi Alexandre,

     

    Thank you for your email. From the Running a Test Case article: 

    The Status of the test case will be set to FAILED if one of the following conditions is met:

    • The test case was aborted due to an internal error.

    • The test case was aborted because of a failed test step, or if there were failed test steps and the Fail Test Case on Error option in the TestCase Options dialog is selected.

    • A script in any of the triggered events, scripts, or handlers failed the test case with testRunner.failexplicitly.

    You can find the final state of the test case in the Run Log.

     

    So, please check all these conditions.

    • krogold's avatar
      krogold
      Regular Contributor
      8 years ago

      Indeed, I chose to set the "fail testCase if it has failed TestSteps" in order to detect when a test is failing but, actually, at the end of the test, none of my teststeps is "failed" and I don't have any Error log information.