Forum Discussion

pb1's avatar
pb1
Contributor
4 years ago
Solved

Retry based on AssertionStep status

Hi there   Intro here, the actual problem is below the line. We are currently working on a retry-mechanism that re-runs a specific interval of testSteps upon failure.    In the given pictur...
Assertions
Scripting
  • pb1's avatar
    pb1
    4 years ago

    I managed to solve this specific problem, the opened support request shall remain open because I still suspect a problem with the .run() function.

     

    Instead of trying to emulate the testRunner from inside the groovy step we now "drop an anchor" before the retry-cycle, run into the script and check the checkpoint status. If the status is FAIL we tell the testRunner to go back to the anchor via testRunner.gotoStep() - if the status is PASS we just let the TestCase continue on. Here is how that looks:

    The anchor contains the number of remaining retry attempts, which are subtracted with each iteration by the ftpRetry-Script.

    And here is the adapted ftpRetry function from the example.

     

    void ftpRetry(def testRunner, def context){
  // variables to allow user to specify other names
  List<String> retryCheckpointNames = ['retryCheckpoint']
  List<String> retryAnchorNames = ['retryAnchor']
  int defaultRetryCount = 15
  ( checkpointStep, checkpointIndex ) = findStepInTestCase(context, true, retryCheckpointNames)
  ( anchorStep, anchorIndex ) = findStepInTestCase(context, true, retryAnchorNames)
  if(checkpointStep.getAssertionStatus().toString() != 'PASS') {
    // checkpoint did not pass, get the anchor, set the testRunner to the index of the anchor and subtract a retry from the anchor, then let everything run until this script is called again
    int retries = anchorStep.getPropertyValue('retries').toInteger()
    if(retries-- > 0) {
      anchorStep.setPropertyValue('retries', retries.toString())
      testRunner.gotoStep(anchorIndex + 1)
    }
    else {
      anchorStep.setPropertyValue('retries', defaultRetryCount.toString())
      assert false : "Out of retries, checkpoint-assertion failed every time!"
    }
  }
  else {
    log.info("Checkpoint passed, resetting retries on anchor.")
    anchorStep.setPropertyValue('retries', defaultRetryCount.toString())
  }
}

     

     

    Should have reconsidered my approach to the problem rather than trying to force the script to emulate the testRunner, oh well.

HimanshuTayal's avatar
HimanshuTayal
Community Hero

pb1 : Write like below

 

runner = step.run(testRunner,context);
log.info runner.getStatus()

 

Hope this will work 🙂

pb1's avatar
pb1
Contributor
4 years ago

HimanshuTayal 

Thanks for the help so far, but sadly your code produces a "PASS" as well.

  • pb1's avatar
    pb1
    Contributor
    4 years ago

    At this point I am 100% sure that calling .run() on the AssertionTestStep does not execute any AssertionEntry. 

    I can not figure out how to emulate the actual testRunner, my best attempt to run the entries follows: 

    def assertionStep = context.getTestCase().getTestStepAt(context.getCurrentStepIndex() + 4)
assertionStep.getAssertionEntryList()
	.each{ assertionEntry ->
		assertionEntry.getAssertion().assertRequest(?????, context)
	}.collect { assertionEntry ->
		assertionEntry.getStatus()
	}

    The problem right now is that assertRequest() requires the MessageExchange object, but I have no idea how to get the MessageExchange from the previous RestRequest, the API is very undocumented and after a day of digging I could not find a way to make it work.

    • sonya_m's avatar
      sonya_m
      SmartBear Alumni (Retired)
      4 years ago

      pb1 I see that you are working with Support on this (case 00448152).

      Please keep the investigation going. Also, we would really appreciate it if you post your solution here that you get from Support. Thank you.

      • pb1's avatar
        pb1
        Contributor
        4 years ago

        I managed to solve this specific problem, the opened support request shall remain open because I still suspect a problem with the .run() function.

         

        Instead of trying to emulate the testRunner from inside the groovy step we now "drop an anchor" before the retry-cycle, run into the script and check the checkpoint status. If the status is FAIL we tell the testRunner to go back to the anchor via testRunner.gotoStep() - if the status is PASS we just let the TestCase continue on. Here is how that looks:

        The anchor contains the number of remaining retry attempts, which are subtracted with each iteration by the ftpRetry-Script.

        And here is the adapted ftpRetry function from the example.

         

        void ftpRetry(def testRunner, def context){
  // variables to allow user to specify other names
  List<String> retryCheckpointNames = ['retryCheckpoint']
  List<String> retryAnchorNames = ['retryAnchor']
  int defaultRetryCount = 15
  ( checkpointStep, checkpointIndex ) = findStepInTestCase(context, true, retryCheckpointNames)
  ( anchorStep, anchorIndex ) = findStepInTestCase(context, true, retryAnchorNames)
  if(checkpointStep.getAssertionStatus().toString() != 'PASS') {
    // checkpoint did not pass, get the anchor, set the testRunner to the index of the anchor and subtract a retry from the anchor, then let everything run until this script is called again
    int retries = anchorStep.getPropertyValue('retries').toInteger()
    if(retries-- > 0) {
      anchorStep.setPropertyValue('retries', retries.toString())
      testRunner.gotoStep(anchorIndex + 1)
    }
    else {
      anchorStep.setPropertyValue('retries', defaultRetryCount.toString())
      assert false : "Out of retries, checkpoint-assertion failed every time!"
    }
  }
  else {
    log.info("Checkpoint passed, resetting retries on anchor.")
    anchorStep.setPropertyValue('retries', defaultRetryCount.toString())
  }
}

         

         

        Should have reconsidered my approach to the problem rather than trying to force the script to emulate the testRunner, oh well.