Reply
Highlighted
Community Expert
Posts: 540
Registered: ‎12-10-2010

[Resolved] running testCase from testSuite teardown

I have a somewhat complicated test - testing for a race condition - so I have several testcase in a testsuite all run in parallel. The last testcase - one that verifies all the results - in the suite needs to be run only after all the other testcases are done.

I have achieved this by having my last verification case disabled, and I run it from the testsuite teardown script with:
testSuite.getTestCaseByName("verification").run((com.eviware.soapui.support.types.StringToObjectMap)context, true)

What does the second (boolean) parameter do?

If I boil down my verification testcase again to the bare minimum (for the purposes of this discussion) I have just a Groovy script:

try {
assert false : "failed assert"
} catch(Exception) {
testRunner.fail("failed testrunner")
}


When I run this setup from the GUI, the testcase fails (turn red), however the testsuite passes (stays green). How do I get the testsuite to "fail"?

When I run this setup from the command line testrunner, everything passes and I do not see either of the above messages. How do I get the testsuite to "fail"? Where should these two message be logged?
Staff
Posts: 8,810
Registered: ‎02-16-2009

Re: running testCase from testSuite teardown

I wouldn't do this in a tearDown, but use a Groovy script and the TestMonitor class. I think this would work well for you:

import com.eviware.soapui.monitor.TestMonitor

def maxSecondsToWait = 10 // or whatever value of your choosing
while (TestMonitor.hasRunningTests(testSuite.project) && retries <= maxSecondsToWait) {
try { Thread.sleep(1000) } catch (InterruptedException e) {}
}
// verify!
Community Expert
Posts: 540
Registered: ‎12-10-2010

Re: running testCase from testSuite teardown

Nope

Looking at the API here: http://www.soapui.org/apidocs/index.htm ... nitor.html (would be really kool to have some documentation there )

import com.eviware.soapui.monitor.TestMonitor
TestMonitor.hasRunningTests()

gives me
Wed Oct 16 08:16:40 PDT 2013:ERROR:groovy.lang.MissingMethodException: No signature of method: static com.eviware.soapui.monitor.TestMonitor.hasRunningTests() is applicable for argument types: () values: []
Possible solutions: hasRunningTests(), hasRunningTests(com.eviware.soapui.impl.wsdl.WsdlProject), hasRunningTest(com.eviware.soapui.model.testsuite.TestCase), hasRunningTestCase(com.eviware.soapui.model.testsuite.TestCase)


Since a Groovy Step does not have the property testSuite, I tried:
import com.eviware.soapui.monitor.TestMonitor
TestMonitor.hasRunningTests(testRunner.testCase.testSuite.project)

which gives me:
Wed Oct 16 08:18:33 PDT 2013:ERROR:groovy.lang.MissingMethodException: No signature of method: static com.eviware.soapui.monitor.TestMonitor.hasRunningTests() is applicable for argument types: (com.eviware.soapui.impl.wsdl.WsdlProjectPro) values: [com.eviware.soapui.impl.wsdl.WsdlProjectPro@45ea368b]
Possible solutions: hasRunningTests(), hasRunningTests(com.eviware.soapui.impl.wsdl.WsdlProject), hasRunningTest(com.eviware.soapui.model.testsuite.TestCase), hasRunningTestCase(com.eviware.soapui.model.testsuite.TestCase)

Does this mean I have to downgrade to the non-Pro version?
Community Expert
Posts: 540
Registered: ‎12-10-2010

Re: running testCase from testSuite teardown

The secret make it work switch is:
def tc = new com.eviware.soapui.monitor.TestMonitor
tc.hasRunningTests(...)

Community Expert
Posts: 540
Registered: ‎12-10-2010

Re: running testCase from testSuite teardown

I do not think this works.

1. Am I allowed to run this code from the testCase Setup Script? I tried it both from the Setup Script as well as a Groovy testStep.

2. In the while loop I inserted a log.info("I wuz here!"); this message never gets logged! Why not?

3. In the code sample you provided what is "retries"?

4. In the code sample you provided what is the point of the try..catch?
Staff
Posts: 8,810
Registered: ‎02-16-2009

Re: running testCase from testSuite teardown

Hi!

Yes, you should be able to run this from the Setup Script as well as a Groovy test step.
Sorry for the code being a bit incomplete. The following code probably works better:


import com.eviware.soapui.monitor.TestMonitor

def monitor = new com.eviware.soapui.monitor.TestMonitor()
def maxSecondsToWait = 10 // or whatever value of your choosing
def retry = 0
while ( monitor.hasRunningTests() && retry < maxSecondsToWait) {
Thread.sleep(1000)
retry++
}


To answer your other questions,
2. Probably because "retries" was never initialized and not incremented in the loop
3. It's just a counter that is incremented for each loop
4. try..catch is needed in java because Thread.sleep() throws a checked exception, but it is not needed in groovy.

Hope this helps,
Anders
SmartBear Sweden
Community Expert
Posts: 540
Registered: ‎12-10-2010

Re: running testCase from testSuite teardown

Now it starts to get complicated. I have two test cases in a test suite, mode=PARALLEL.

TestCase 1.
1. SOAP call to a service
2. Transfer step to retrieve a value from #1 to a TestSuite property "startingValue". This property has value "empty" at the start of test.

TestCase 2.
1. Groovy Script

import com.eviware.soapui.monitor.TestMonitor

def monitor = new com.eviware.soapui.monitor.TestMonitor()
while(monitor.hasRunningTest("TestCase 1.")) {
log.info "I wuz here!"
Thread.sleep(100)
}

2. SOAP call to a service that uses ${#TestSuite#startingValue}

In my last step above, I always see the value "empty" when I look at the Raw Request of the SOAP call, however I see the value updated in the testSuite property when the test is finished running. Also, the info message never gets logged.

I also tried hasRunningTestCase() instead of hasRunningTest(); is there a difference?

I did not use hasRunningTests(), because in my situation as described here that seems like it is going to be asking: am I running while I am running?

What am I doing wrong?
Staff
Posts: 8,810
Registered: ‎02-16-2009

Re: running testCase from testSuite teardown

Hi,

It seems as if the TestMonitor solution didn't work as we hoped. We're trying to figure out another solution, and we'll get back to you as soon as possible.

Regards,
Arian
SmartBear Sweden
Community Expert
Posts: 540
Registered: ‎12-10-2010

Re: [Open] running testCase from testSuite teardown

Staff
Posts: 8,810
Registered: ‎02-16-2009

Re: [Open] running testCase from testSuite teardown

Hello again,

I've come up with a solution. I need to tell you that this is really outside the scope of the support offering - we generally don't support Groovy scripts. However, I decided to make an exception for you because my initial suggestion was messed up - sorry about that - and also because your problem piqued my interest so I looked at it in my own time. :-)

Solution that seems to work for me:

1. Add a test case first in your test suite, with a single Groovy test step containing the following code:
def monitor = new com.eviware.soapui.monitor.TestMonitor()
def thisTestCase = context.testCase
thisTestCase.testSuite.testCases.each {
testCase = it.value
if (testCase != thisTestCase && testCase.label != "Wait for completion") {
log.info("Monitoring $testCase.label")
monitor.monitorTestCase(testCase)
}
}
context["#TestSuiteRunner#"].runContext.testMonitor = monitor

2. Create another test case at the end of the suite, with the label "Wait for completion" and the following code:
def monitor = context["#TestSuiteRunner#"].runContext.testMonitor
if (monitor == null) {
log.warn("Monitor not found in completion check")
}
else {
def maxWaitTimeInSeconds = 10
def retries = 0
while (monitor.hasRunningTests() && retries < maxWaitTimeInSeconds) {
log.info("Waiting for tests to complete ...")
try {
Thread.sleep(1000)
}
catch (InterruptedException ignored) { }
retries++
}
log.info("Tests completed: " + !monitor.hasRunningTests())
}


... and then obviously replace the final log statements with the checks you want to perform. By doing this you should be able to fail the whole suite properly.

Hope this helps,
Manne