Forum Discussion

Flib's avatar
Flib
New Contributor
10 months ago

Rerun TestCase don't save new result in ReadyApi report

Hello,

I see previous subjects on this problem, but they aren't resolved. 

I have a test suite with the ready API rerun script :

import com.eviware.soapui.model.testsuite.TestRunner.Status

//Rerun failed test cases X amount of times based off the number set in the RerunCount test suite property
def reRunCount = Integer.parseInt(testSuite.getPropertyValue("RerunCount"))
def failTestSuite = false

for (testCaseResult in runner.results) {
    if (testCaseResult.getStatus().toString() == 'FAIL') {
             def tRun 
        for (i = 0; i < reRunCount; i++) {
            tRun = testCaseResult.testCase.run(null, false) //Need to have "true" as the second argument
            log.info("Run : ." + testCaseResult.getTestCase().name)

            if (tRun.getStatus().toString() == "PASS") {
                // Set the status of the testCaseResult to PASS if the rerun is successful
                testCaseResult.setStatus(Status.FINISHED)
                runner.status = Status.FINISHED // Change test suite status to PASS
                break;
            }
        }
        if (tRun.getStatus().toString() == "FAILED") {
            failTestSuite = true // If a test case is still failed after reruns, set the flag to fail the test suite and exit the loop
        }
    }
    log.info testCaseResult.getStatus().toString()
}

if (failTestSuite) {
    runner.status = Status.FAILED // Change the test suite status to failed if the flag is set to fail the test suite
}

When a test case is FAIL, the script work well and rerun it. But if the rerun is PASS, the XML report (TestSuiteTestCaseResults) still write it as FAIL but with all testSteps PASS.

Exemple : 

<?xml version="1.0" encoding="UTF-8"?>
<testSuiteResults>
  <testCase>
    <reason>Failing due to failed test step</reason>
    <timeTaken>1715</timeTaken>
    <startTime>17:17:27</startTime>
    <testCaseName>Complement reorientation</testCaseName>
    <status>FAIL</status>
    <testCaseId>92a792b0-b9f0-4579-bb54-2285668610e7</testCaseId>
    <testStepParameters>
      <parameters>
        <iconPath>/rest_request_step.svg</iconPath>
        <testStepName>Modification commentaire</testStepName>
      </parameters>
      <parameters>
        <iconPath>/rest_request_step.svg</iconPath>
        <testStepName>Modification Mouvement réo</testStepName>
      </parameters>
    </testStepParameters>
    <failedTestSteps/>
    <testStepResults>
      <result>
        <timeTaken>547</timeTaken>
        <name>Modification commentaire</name>
        <started>17:17:29.969</started>
        <message>Step 1 [Modification commentaire] PASS: took 547 ms</message>
        <order>2</order>
        <status>PASS</status>
      </result>
      <result>
        <timeTaken>1068</timeTaken>
        <name>Modification Mouvement réo</name>
        <started>17:17:30.592</started>
        <message>Step 2 [Modification Mouvement réo] PASS: took 1068 ms</message>
        <order>3</order>
        <status>PASS</status>
      </result>
    </testStepResults>
  </testCase>
</testSuiteResults>

As you can see, all testSteps are PASS, but testCase status is FAIL. 

I modify the script to add this line testCaseResult.setStatus(Status.FINISHED) and change testCase status from FAIL to PASS. But the report still don't get it. 

Any idea ? 

  • Flib's avatar
    Flib
    New Contributor

    Nota : I have the same behavior manually : 

    If I run testSuite "Testage" and fail a step in "complement reorientation", then I rerun only testCase "complement reorientation" and pass it. In the report,  "complement reorientation" is still tagged as FAIL. 

    I think, I had to change the teardown script to run the testSuite "Testage", and automatically pass all testCase if they are already PASS in the runner. 

    I check if it's possible