Forum Discussion

Nagravision_SA__1's avatar
12 years ago

Loop on AssertionStatus of a REST GET Call not working

Hello,

I wrote a script to loop on a given testStep until the assertionStatus is valid or a specified time out occur.
In this case, I am checking a on GET REST Response until a status has been reached. The Mockup Service reply the content of a property.

- If I give the correct response and run the script, the AssertionStatus return VALID and I leave the loop succesfully.
- If i give the incorrect response and rune the script, the Loop check periodically for the response. I the supply the correct response in the property, but the AssertionStatus remains FAILED.
- if right after I stop the script and run it again .. it works properly.

Can you advise whyn the AssertionStatus is not set properly in the loop ??

The same problem is true if instead of calling a mock service, i am calling a web service (SOAP) waiting for a STARTED status to be returned (basically telling me the server is up and running).
Loop send me UNKNOWN assert status, since the web service is not available to start with, but never get me the VALID status, even after the service is up and running...



import com.eviware.soapui.model.testsuite.Assertable.AssertionStatus;
import org.apache.commons.lang.time.StopWatch
def long timeout = 0
def String callingTestSuite
def String callingTestCase
def testStepToRun
boolean success = false
boolean wait = true
// Initialising external properties
try {
timeout = testRunner.testCase.getPropertyValue("timeOutInMs").toLong()
} catch (Throwable e) {
log.error('Missing timeOutInMs Property. Please check Testcase '+testRunner.testCase.getLabel())
assert false, 'Missing timeOutInMs Property. Please check Testcase '+testRunner.testCase.getLabel()
}

try {
callingTestSuite = testRunner.testCase.getPropertyValue("callingTestSuite")
} catch (Throwable e) {
log.error('Missing callingTestSuite Property. Please check Testcase '+testRunner.testCase.getLabel())
assert false, 'Missing callingTestSuite Property. Please check Testcase '+testRunner.testCase.getLabel()
}

try {
callingTestCase = testRunner.testCase.getPropertyValue("callingTestCase")
} catch (Throwable e) {
log.error('Missing callingTestCase Property. Please check Testcase '+testRunner.testCase.getLabel())
assert false, 'Missing callingTestCase Property. Please check Testcase '+testRunner.testCase.getLabel()
}

try {
stepName = testRunner.testCase.getPropertyValue("testStepNameToRun")
} catch (Throwable e) {
log.error('Missing testStepNameToRun Property. Please check Testcase '+testRunner.testCase.getLabel())
assert false, 'Missing testStepNameToRun Property. Please check Testcase '+testRunner.testCase.getLabel()
}

if (stepName != null) {
try {
testStepToRun = testRunner.testCase.getProject().getTestSuiteByName(callingTestSuite).getTestCaseByName(callingTestCase).getTestStepByName(stepName)
} catch (Throwable e) {
log.error('Could not found test Step named '+ stepName +' in '+ testRunner.testCase.getProject().getName() +'/'+ callingTestSuite + '/' + callingTestCase)
assert false, 'Could not found test Step named '+ stepName +' in '+ testRunner.testCase.getProject().getName() +'/'+ callingTestSuite + '/' + callingTestCase
}
// Initialise stopWatch
StopWatch stopWatch = new StopWatch()
stopWatch.start()
log.info ('Calling ' + testStepToRun.getLabel() + ' until timeout (' +timeout+ 'ms) or success')
while(!(success) && wait) {
// Rune the provided test step
testRunner.runTestStep(testStepToRun)
// Check assertion status
success = testStepToRun.getAssertionStatus() == AssertionStatus.VALID
log.info "[LOOP] Assert Status :"+ testStepToRun.getAssertionStatus()
wait = stopWatch.getTime() <= timeout
if(!wait) {
log.error(stepName +" could not be treated in the given timeout (${timeout}) ms.")
}
if(success) {
log.info(stepName +" finished succesfully.")
}
sleep(1000)
}
stopWatch.stop()
} else {
log.error('The stepName property cannot be null when calling API method '+testRunner.testCase.getProject().getName() +'/'+ callingTestSuite + '/' + callingTestCase)
assert false, 'The stepName property cannot be null when calling API method '+testRunner.testCase.getProject().getName() +'/'+ callingTestSuite + '/' + callingTestCase
}

8 Replies

  • The script is in a GroovyScript Test Step of a generic Test Case.
    Another test case, with the WebServiceCall and Assertion is calling upon it passing it:
    - the Timeout
    - which Test Step to Run (thw web service call)
    - And the TestSuite / TestCase calling the script (required to get the TestStep by name)



    In essence its a very simple script that we want to use in several places in WebService test strategy.
    You can reproduce this easily by caling upion a web service and adding any assertion you want, that would be valid as soon as the web service is responding.
    Do not start the web service, run your loop with a high enough timeout, the AssertionStatus will be returned as UNKOWN.
    Start your web service, and it wont change.
    An individual call to the same web service however would return an AssertionStatus of VALID...
  • Hello,

    I looked at the handler, but I cannot see how this would help in our case.
    The LoopUntilTimeOutOrAssert is an utility TestCase. It is generic for any kind of SOAP or REST call.

    It is then called by many Test case, for which we physically create the GET request and add assertion to it. This GET Request is disabled in the test case, but passed to the LoopUntilTimeOutOrAssert script.
    The script is in charge of running the specified GET request, and check its assertion results.


    If I move the script in an event handler:
    - each new GET response it should apply to must be added to the filter, or use the same naming convention -> DoAble
    - How does I get the assertion in the afterRequest handler ? I have the request available at this level, but getAssertionStatus() is available on the RestTestRequestStep...
    - I guess if i managed to get the assertion I would then need to set a property with the status to test it in the LoopUntilTimeOutOrAssert script?


    In any case... I still fail to understand why the getAssertionStatus() is not correct in my loop. Is this a known bug of the API, if not can it be entered as a bug and dealt with accordingly?
  • I have added a log into the aftterRequest event.
    If I run the GET test step manually.. No problem, the even is reached and the log displayed.
    If i run the script and use the run() method, the log is never displayed....
  • This is indeed what we have done has a workaround for now.
    BUT:
    - this does not solve the bug. Why is it not working when calling the run() method on the test step ?
    - It is not as gracious as we wanted to create this. We are trying to provide "utilities" test cases that can be reused. Using GOTO step is more messy and is not re-usable.
    - Using GOT step, it is not possible to also check for TimeOut. Our goal is to achieve a SUCCESS or TIMEOUT method on a REST/SOAP call.

    Can you please let us know if this is an identified bug on your side and when it will be fixed please ?
  • Hi,

    Try this and see if it helps.


    //testRunner.runTestStep(testStepToRun) replace this with
    testStepToRun.run(testrunner,context)



    Regards,
    Marcus
    SmartBear Support