Forum Discussion

krogold's avatar
krogold
Regular Contributor
8 years ago

Correct test Case execution gives a 'Failed' status

Hello,   I have a strange problem with testCase execution. I have a test case with the following structure: Setup script:   set session_id = 0   set session_index = 0   store custom properties...
  • krogold's avatar
    krogold
    8 years ago

    Finally I found a workaround for this problem:

    I checked the results and found out that some of my test steps where considered as failed in testRunner.results. I set the following bit of code in my teardown script tab

     

    for (testStep in testRunner.getResults()){
	log.info "status " + testStep.getTestStep().getName() + " : " + testStep.getStatus()
}

    These steps were the ones on which I was looping, waiting for the status different from 'pending'. At the end of the test, as the status was finally 'success' the step was set as OK in the testCase (green-flagged), even if several occurrences of the testStep were 'failed'

    So I found I could update those status doing the following for the concerned steps:

    for (results in testRunner.results){
     // selection condition
        results.status = "OK"
    }

    I just have to make sure I apply this only on the concerned testSteps.

     

    But at the end the overall status is still FAILED, still in my teardown script:

    log.info "TEST RUNNER STATUS after update" + testRunner.getStatus()

    If I'm sure of my steps and result, I can overwrite it as follows:

    testRunner.status = "FINISHED"
log.info "TEST RUNNER STATUS after update" + testRunner.getStatus

    And my overall test is ok (green bar)

    I know it's a bit tricky, but as long as I set the proper conditions to update those parameters, I'm sure not to hide real failures.

     

     

Nastya_Khovrina's avatar
Nastya_Khovrina
SmartBear Alumni (Retired)

Hi Alexandre,

 

Thank you for your email. From the Running a Test Case article: 

The Status of the test case will be set to FAILED if one of the following conditions is met:

  • The test case was aborted due to an internal error.

  • The test case was aborted because of a failed test step, or if there were failed test steps and the Fail Test Case on Error option in the TestCase Options dialog is selected.

  • A script in any of the triggered events, scripts, or handlers failed the test case with testRunner.failexplicitly.

You can find the final state of the test case in the Run Log.

 

So, please check all these conditions.

krogold's avatar
krogold
Regular Contributor
8 years ago

Indeed, I chose to set the "fail testCase if it has failed TestSteps" in order to detect when a test is failing but, actually, at the end of the test, none of my teststeps is "failed" and I don't have any Error log information.

    • krogold's avatar
      krogold
      Regular Contributor
      8 years ago

      ok, I'll do that

       

      thank you

       

       

      • krogold's avatar
        krogold
        Regular Contributor
        8 years ago

        Finally I found a workaround for this problem:

        I checked the results and found out that some of my test steps where considered as failed in testRunner.results. I set the following bit of code in my teardown script tab

         

        for (testStep in testRunner.getResults()){
	log.info "status " + testStep.getTestStep().getName() + " : " + testStep.getStatus()
}

        These steps were the ones on which I was looping, waiting for the status different from 'pending'. At the end of the test, as the status was finally 'success' the step was set as OK in the testCase (green-flagged), even if several occurrences of the testStep were 'failed'

        So I found I could update those status doing the following for the concerned steps:

        for (results in testRunner.results){
         // selection condition
            results.status = "OK"
        }

        I just have to make sure I apply this only on the concerned testSteps.

         

        But at the end the overall status is still FAILED, still in my teardown script:

        log.info "TEST RUNNER STATUS after update" + testRunner.getStatus()

        If I'm sure of my steps and result, I can overwrite it as follows:

        testRunner.status = "FINISHED"
log.info "TEST RUNNER STATUS after update" + testRunner.getStatus

        And my overall test is ok (green bar)

        I know it's a bit tricky, but as long as I set the proper conditions to update those parameters, I'm sure not to hide real failures.