tc.run() does not function in load tests

Question

SoapUI v.4.6.1 x64 (the newest, but the same was in 4.6.0) on Windows 7 x64.

I have a simple sample (see attachments) demonstrating that running another TestCase from groovy script via tc.run() behaves strangely in load tests. Strange groovy errors occur even in the simplest situation, and moreover the execution environment becomes unstable (in a sense).

I have a TestCase containing this groovy script:

def tc = testRunner.testCase.testSuite.testCases["TestCase - Proc"]
def runner = tc.run(null, false )

The other TestCase "TestCase - Proc" contains this groovy script:

def c = context

That's all.

1. When executed from a load test in at least two threads, the following errors occur in the error log in about 3% of Total Runs:

Sat Nov 02 18:40:37 CET 2013:ERROR:groovy.lang.MissingPropertyException: No such property: context for class: Script1
groovy.lang.MissingPropertyException: No such property: context for class: Script1
at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.unwrap(ScriptBytecodeAdapter.java:50)
at org.codehaus.groovy.runtime.callsite.PogoGetPropertySite.getProperty(PogoGetPropertySite.java:49)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callGroovyObjectGetProperty(AbstractCallSite.java:231)
at Script1.run(Script1.groovy:1)
at com.eviware.soapui.support.scripting.groovy.SoapUIGroovyScriptEngine.run(SoapUIGroovyScriptEngine.java:96)
at com.eviware.soapui.impl.wsdl.teststeps.WsdlGroovyScriptTestStep.run(WsdlGroovyScriptTestStep.java:149)
at com.eviware.soapui.impl.wsdl.support.AbstractTestCaseRunner.runTestStep(AbstractTestCaseRunner.java:239)
at com.eviware.soapui.impl.wsdl.testcase.WsdlTestCaseRunner.runCurrentTestStep(WsdlTestCaseRunner.java:48)
at com.eviware.soapui.impl.wsdl.support.AbstractTestCaseRunner.internalRun(AbstractTestCaseRunner.java:148)
at com.eviware.soapui.impl.wsdl.support.AbstractTestCaseRunner.internalRun(AbstractTestCaseRunner.java:43)
at com.eviware.soapui.impl.wsdl.support.AbstractTestRunner.run(AbstractTestRunner.java:135)
at com.eviware.soapui.impl.wsdl.support.AbstractTestRunner.start(AbstractTestRunner.java:77)
at com.eviware.soapui.impl.wsdl.testcase.WsdlTestCase.run(WsdlTestCase.java:640)
at com.eviware.soapui.impl.wsdl.testcase.WsdlTestCase.run(WsdlTestCase.java:73)
at com.eviware.soapui.model.testsuite.TestCase$run.call(Unknown Source)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:42)
at com.eviware.soapui.model.testsuite.TestCase$run.call(Unknown Source)
at Script1.run(Script1.groovy:4)
at com.eviware.soapui.support.scripting.groovy.SoapUIGroovyScriptEngine.run(SoapUIGroovyScriptEngine.java:96)
at com.eviware.soapui.impl.wsdl.teststeps.WsdlGroovyScriptTestStep.run(WsdlGroovyScriptTestStep.java:149)
at com.eviware.soapui.impl.wsdl.support.AbstractTestCaseRunner.runTestStep(AbstractTestCaseRunner.java:239)
at com.eviware.soapui.impl.wsdl.testcase.WsdlTestCaseRunner.runCurrentTestStep(WsdlTestCaseRunner.java:48)
at com.eviware.soapui.impl.wsdl.support.AbstractTestCaseRunner.internalRun(AbstractTestCaseRunner.java:148)
at com.eviware.soapui.impl.wsdl.support.AbstractTestCaseRunner.internalRun(AbstractTestCaseRunner.java:43)
at com.eviware.soapui.impl.wsdl.support.AbstractTestRunner.run(AbstractTestRunner.java:135)
at com.eviware.soapui.impl.wsdl.loadtest.WsdlLoadTestRunner$InternalTestCaseRunner.run(WsdlLoadTestRunner.java:544)
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)

2. When executing the load test for several times (and making some editations in between, maybe), the error log contains this error instead:

Sat Nov 02 18:38:53 CET 2013:ERROR:java.lang.IndexOutOfBoundsException: Index: 1, Size: 1
java.lang.IndexOutOfBoundsException: Index: 1, Size: 1
at java.util.ArrayList.rangeCheck(Unknown Source)
at java.util.ArrayList.get(Unknown Source)
at com.eviware.soapui.impl.wsdl.panels.support.TestRunComponentEnabler.enable(TestRunComponentEnabler.java:80)
at com.eviware.soapui.impl.wsdl.panels.support.TestRunComponentEnabler.testCaseFinished(TestRunComponentEnabler.java:107)
at com.eviware.soapui.monitor.TestMonitor.notifyTestCaseFinished(TestMonitor.java:154)
at com.eviware.soapui.monitor.TestMonitor$InternalTestRunListener.afterRun(TestMonitor.java:294)
at com.eviware.soapui.impl.wsdl.support.AbstractTestCaseRunner.notifyAfterRun(AbstractTestCaseRunner.java:301)
at com.eviware.soapui.impl.wsdl.support.AbstractTestCaseRunner.internalFinally(AbstractTestCaseRunner.java:191)
at com.eviware.soapui.impl.wsdl.support.AbstractTestCaseRunner.internalFinally(AbstractTestCaseRunner.java:43)
at com.eviware.soapui.impl.wsdl.support.AbstractTestRunner.run(AbstractTestRunner.java:163)
at com.eviware.soapui.impl.wsdl.support.AbstractTestRunner.start(AbstractTestRunner.java:77)
at com.eviware.soapui.impl.wsdl.testcase.WsdlTestCase.run(WsdlTestCase.java:640)
at com.eviware.soapui.impl.wsdl.testcase.WsdlTestCase.run(WsdlTestCase.java:73)
at com.eviware.soapui.model.testsuite.TestCase$run.call(Unknown Source)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:42)
at com.eviware.soapui.model.testsuite.TestCase$run.call(Unknown Source)
at Script1.run(Script1.groovy:4)
at com.eviware.soapui.support.scripting.groovy.SoapUIGroovyScriptEngine.run(SoapUIGroovyScriptEngine.java:96)
at com.eviware.soapui.impl.wsdl.teststeps.WsdlGroovyScriptTestStep.run(WsdlGroovyScriptTestStep.java:149)
at com.eviware.soapui.impl.wsdl.support.AbstractTestCaseRunner.runTestStep(AbstractTestCaseRunner.java:239)
at com.eviware.soapui.impl.wsdl.testcase.WsdlTestCaseRunner.runCurrentTestStep(WsdlTestCaseRunner.java:48)
at com.eviware.soapui.impl.wsdl.support.AbstractTestCaseRunner.internalRun(AbstractTestCaseRunner.java:148)
at com.eviware.soapui.impl.wsdl.support.AbstractTestCaseRunner.internalRun(AbstractTestCaseRunner.java:43)
at com.eviware.soapui.impl.wsdl.support.AbstractTestRunner.run(AbstractTestRunner.java:135)
at com.eviware.soapui.impl.wsdl.loadtest.WsdlLoadTestRunner$InternalTestCaseRunner.run(WsdlLoadTestRunner.java:544)
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)

When running the "TestCase - Proc" from the ordinary Run TestCase step, the errors do not occur.

3. Moreover, after several executions of the load test, I cannot edit the script code of "TestCase - Proc", the groovy script editor is in read-only mode. I have to exit and restart SoapUI after that. The explanation is shown in the SoapUI exit dialog: The TestCase is still considered running ("Exit SoapUI? Projects with running tests will not be saved.")

All in all, tc.run() is currently unusable from load tests.

smartbear_suppo · Answer

We'll have a look at this, but load testing is not a priority feature in SoapUI. We recommend you to use the companion product LoadUI if you want to do serious load testing. It's a full-fledged load testing tool, and just like SoapUI it has a good free version.

Kind regards,
Manne

xarx123 · Answer

I've just try that, the same problem is also in LoadUI. I used the SoapUI Runner with the SoapUI project I had sent you before, and under a sufficiently high load (say 1000 runs/sec) there are the same errors as above in the LoadUI System Log. In my point of view, this is a critical bug that causes the LoadUI with the SoapUI Runner unreliable, and therefore unusable (it's a "show-stopper").

marcusj · Answer

Hi,I was able to reproduce problem 1. you listed "ERROR:groovy.lang.MissingPropertyException: No such property: context for class: Script1 groovy.lang.MissingPropertyException: No such property: context for class: Script1" in SoapUI load test and LoadUI load test.Defect SOAP-965 has been opened to track this.  The two other problems you listed did not occur for me.

siemens__s_r_o_ · Answer

Hey,&nbsp;I know I'm 3 years late here, but I retested this on 5.2.1, still getting the error.&nbsp;Here's the solution:I managed to avoid the issue with synchonized():def tc = testRunner.testCase.testSuite.testCases["callee"]synchronized(tc){ def runner = tc.run(null, false )}Run a couple of times for a minute with Load Test set to Simple Strategy, 0 Test Delay and 0 Random. I got around 5mil executions per minute, zero errors.&nbsp;Same setup, but not synchronized, rolled off 10k rows error log in 3 seconds, getting spammed with the one you got:ERROR:groovy.lang.MissingPropertyException: No such property: context for class: Script1&nbsp;Hope this helps :)&nbsp;Jakub

Forum Discussion

tc.run() does not function in load tests

4 Replies

Related Content

Execute Scenario from AfterFeature function

How to test multi functions

Function to wait for processing to complete

ClickItem is not a function

Log functions

Recent Discussions

How to remove (not clear) test case properties from SaveScript

Remove Properties of TestCase from Groovy.

MyFirstPlugin - button Project menu in Ready Api

Integrate ReadyApi with Devops Release pipelines

Is it possible to integrate the Azure TestPlan Test cases and the ReadyAPI tests ?