Forum Discussion

Elvorin's avatar
Elvorin
Contributor
13 years ago

Not able to generate load properly

Running latest nightly build. Issue is multi-folds -

1. I've set a test case with fixed rate generator. It's supposed to generate 25/sec, with burst size 25 (tried 1 too). But it doesnt send that many requests per second.

2. It starts throwing error pretty soon saying 'java.util.ConcurrentModificationException: Document changed during select'. I have property transfer from the response in my test case, for logging purposes. That is causing the failure (confirmed by enabling/disabling that step).

3. Requests/sec graph shows it going down from 25 to 1 or 2 within few seconds.


So how do I solve these issues? I have to run load test on apps which can take 400req/sec. Please suggest ASAP.

14 Replies

Sort By
Show More
  • SmartBear_Suppo's avatar
    SmartBear_Suppo
    SmartBear Alumni (Retired)
    13 years ago
    Elvorin wrote:
    Seeing a lot of

    "23416","[]","18:54:47:648","java.lang.Exception: Failed to prepare testStep [DataSource]; java.lang.Exception: DataSource not set in step [DataSource]".

    And yes, my datasource is setup. If I run with a small load, it does work without any issue.

    Hi!

    I tried to reproduce this, but everything worked fine for me using loadUI 2.0 beta1 and soapUI 4.5 beta1.

    I need to ask some questions to better help you:


    • What is a small load and what is not?

    • Does this happen when load testing a single TestCase (one soapUI Runner) with high load, or does it just happen with multiple soapUI Runners?

    • If it happens with a single TestCase as well, does it also happen if you load test that TestCase in soapUI alone?

    • Have you tried with loadUI 2.0 (beta 1)?

    • Where are you seeing the error message? Is there more information in C:\Users\UserName\.loadui\logs\soapui-errors.log?


    Thanks

    Henrik
    SmartBear Software
  • Elvorin's avatar
    Elvorin
    Contributor
    13 years ago
    SmartBear Support wrote:
    Elvorin wrote:
    Seeing a lot of

    "23416","[]","18:54:47:648","java.lang.Exception: Failed to prepare testStep [DataSource]; java.lang.Exception: DataSource not set in step [DataSource]".

    And yes, my datasource is setup. If I run with a small load, it does work without any issue.

    Hi!

    I tried to reproduce this, but everything worked fine for me using loadUI 2.0 beta1 and soapUI 4.5 beta1.

    I need to ask some questions to better help you:


    • What is a small load and what is not?

    • Does this happen when load testing a single TestCase (one soapUI Runner) with high load, or does it just happen with multiple soapUI Runners?

    • If it happens with a single TestCase as well, does it also happen if you load test that TestCase in soapUI alone?

    • Have you tried with loadUI 2.0 (beta 1)?

    • Where are you seeing the error message? Is there more information in C:\Users\UserName\.loadui\logs\soapui-errors.log?


    Thanks

    Henrik
    SmartBear Software


    1. Small load is when 10-20 requests/sec; High load is 100 req/sec
    2. Have not tried putting 100 req/sec for a single SoapRunner. But the issue happens when I run multiple SoapRunner. But the error doesnt happen for all of them. It's more for few of them at random. By random I meant the set of SoapRunners failing isnt always fixed set.
    3. I never generated 100 req/sec for a single SoapRunner. So can't answer this one properly. But I've manually executed the SoapUI test cases for which I'm seeing failures, and they passed everytime. May be because in SoapUI I'm just executing them once, and not in loop.
    4. No, I havent tried beta yet.
    5. Error messages are seen in Table Log, as it's attached to the error output.

    Note: I'm running the load test again right now. Lets see if I can reproduce the errors. It seems more sporadic so far.
  • Elvorin's avatar
    Elvorin
    Contributor
    13 years ago
    While running load test I was not seeing any such error for most of the part. Then I opened SOAP UI Pro for fixing a request (not part of test case) in the project being used by LoadUI. After fixing the request I closed SoapUI. Then I went to Results Workbench (which was already opened) to see how things are looking and noticed suddenly all my added statistics are gone from response time graph. Those statistics were added by me manually (to capture the soap call time taken instead of test case time taken). So in the middle of the run I had to add all of the statistics (18 to be specific) back again. Also noticed that one of the soap runner is now failing for almost all of the calls (couldn't match numbers to say 100% of them, as it was generating 5 req/sec) with the previously mentioned datasource exception. Following is the exception stack trace from Java WS console -


    16:29:25,476 ERROR [SoapUI] An error occured [DataSource not set in step [DataSource]], see error log for details
    java.lang.Exception: DataSource not set in step [DataSource]
     at com.eviware.soapui.impl.wsdl.teststeps.WsdlDataSourceTestStep.prepare(SourceFile:409)
     at com.eviware.soapui.impl.wsdl.support.AbstractTestCaseRunner.internalRun(AbstractTestCaseRunner.java:126)
     at com.eviware.soapui.impl.wsdl.support.AbstractTestCaseRunner.internalRun(AbstractTestCaseRunner.java:42)
     at com.eviware.soapui.impl.wsdl.support.AbstractTestRunner.run(AbstractTestRunner.java:135)
     at com.eviware.loadui.components.soapui.SoapUISamplerComponent$SoapUITestCaseRunner.run(SoapUISamplerComponent.java:975)
     at com.eviware.loadui.components.soapui.SoapUISamplerComponent.sample(SoapUISamplerComponent.java:831)
     at com.eviware.loadui.impl.component.categories.RunnerBase.doSample(RunnerBase.java:496)
     at com.eviware.loadui.impl.component.categories.RunnerBase.access$700(RunnerBase.java:70)
     at com.eviware.loadui.impl.component.categories.RunnerBase$Worker.run(RunnerBase.java:773)
     at com.eviware.loadui.util.dispatch.CustomThreadPoolExecutor$Worker.run(CustomThreadPoolExecutor.java:188)
     at java.lang.Thread.run(Unknown Source)


    I also noticed the same stack trace in soapui-error.log file. Following is what I found in soapui.log file -


    2012-03-05 16:25:35,939 ERROR [AbstractTestRunner] Exception during Test Execution
    java.lang.Exception: Failed to prepare testStep [DataSource]; java.lang.Exception: DataSource not set in step [DataSource]
     at com.eviware.soapui.impl.wsdl.support.AbstractTestCaseRunner.internalRun(AbstractTestCaseRunner.java:132)
     at com.eviware.soapui.impl.wsdl.support.AbstractTestCaseRunner.internalRun(AbstractTestCaseRunner.java:42)
     at com.eviware.soapui.impl.wsdl.support.AbstractTestRunner.run(AbstractTestRunner.java:135)
     at com.eviware.loadui.components.soapui.SoapUISamplerComponent$SoapUITestCaseRunner.run(SoapUISamplerComponent.java:975)
     at com.eviware.loadui.components.soapui.SoapUISamplerComponent.sample(SoapUISamplerComponent.java:831)
     at com.eviware.loadui.impl.component.categories.RunnerBase.doSample(RunnerBase.java:496)
     at com.eviware.loadui.impl.component.categories.RunnerBase.access$700(RunnerBase.java:70)
     at com.eviware.loadui.impl.component.categories.RunnerBase$Worker.run(RunnerBase.java:773)
     at com.eviware.loadui.util.dispatch.CustomThreadPoolExecutor$Worker.run(CustomThreadPoolExecutor.java:188)
     at java.lang.Thread.run(Unknown Source)
    2012-03-05 16:25:36,599 ERROR [SoapUI] An error occured [DataSource not set in step [DataSource]], see error log for details
    2012-03-05 16:25:36,599 ERROR [errorlog] java.lang.Exception: DataSource not set in step [DataSource]
    java.lang.Exception: DataSource not set in step [DataSource]
     at com.eviware.soapui.impl.wsdl.teststeps.WsdlDataSourceTestStep.prepare(SourceFile:409)
     at com.eviware.soapui.impl.wsdl.support.AbstractTestCaseRunner.internalRun(AbstractTestCaseRunner.java:126)
     at com.eviware.soapui.impl.wsdl.support.AbstractTestCaseRunner.internalRun(AbstractTestCaseRunner.java:42)
     at com.eviware.soapui.impl.wsdl.support.AbstractTestRunner.run(AbstractTestRunner.java:135)
     at com.eviware.loadui.components.soapui.SoapUISamplerComponent$SoapUITestCaseRunner.run(SoapUISamplerComponent.java:975)
     at com.eviware.loadui.components.soapui.SoapUISamplerComponent.sample(SoapUISamplerComponent.java:831)
     at com.eviware.loadui.impl.component.categories.RunnerBase.doSample(RunnerBase.java:496)
     at com.eviware.loadui.impl.component.categories.RunnerBase.access$700(RunnerBase.java:70)
     at com.eviware.loadui.impl.component.categories.RunnerBase$Worker.run(RunnerBase.java:773)
     at com.eviware.loadui.util.dispatch.CustomThreadPoolExecutor$Worker.run(CustomThreadPoolExecutor.java:188)
     at java.lang.Thread.run(Unknown Source)


    Additionally I also noticed following errors at the very beginning of my load run -

    5 Times -

    2012-03-05 15:02:22,395 ERROR [errorlog] java.io.IOException: Stream closed
    java.io.IOException: Stream closed
     at java.io.BufferedReader.ensureOpen(Unknown Source)
     at java.io.BufferedReader.readLine(Unknown Source)
     at java.io.BufferedReader.readLine(Unknown Source)
     at com.eviware.soapui.impl.wsdl.teststeps.datasource.FileDataSource.next(SourceFile:150)
     at com.eviware.soapui.impl.wsdl.teststeps.WsdlDataSourceTestStep.next(SourceFile:524)
     at com.eviware.soapui.impl.wsdl.teststeps.WsdlDataSourceTestStep.run(SourceFile:136)
     at com.eviware.soapui.impl.wsdl.support.AbstractTestCaseRunner.runTestStep(AbstractTestCaseRunner.java:232)
     at com.eviware.soapui.impl.wsdl.testcase.WsdlTestCaseRunner.runCurrentTestStep(WsdlTestCaseRunner.java:48)
     at com.eviware.soapui.impl.wsdl.support.AbstractTestCaseRunner.internalRun(AbstractTestCaseRunner.java:141)
     at com.eviware.soapui.impl.wsdl.support.AbstractTestCaseRunner.internalRun(AbstractTestCaseRunner.java:42)
     at com.eviware.soapui.impl.wsdl.support.AbstractTestRunner.run(AbstractTestRunner.java:135)
     at com.eviware.loadui.components.soapui.SoapUISamplerComponent$SoapUITestCaseRunner.run(SoapUISamplerComponent.java:975)
     at com.eviware.loadui.components.soapui.SoapUISamplerComponent.sample(SoapUISamplerComponent.java:831)
     at com.eviware.loadui.impl.component.categories.RunnerBase.doSample(RunnerBase.java:496)
     at com.eviware.loadui.impl.component.categories.RunnerBase.access$700(RunnerBase.java:70)
     at com.eviware.loadui.impl.component.categories.RunnerBase$Worker.run(RunnerBase.java:773)
     at com.eviware.loadui.util.dispatch.CustomThreadPoolExecutor$Worker.run(CustomThreadPoolExecutor.java:188)
     at java.lang.Thread.run(Unknown Source)


    And Once (1) -

    2012-03-05 16:11:12,637 ERROR [errorlog] java.util.ConcurrentModificationException: Document changed during select
    java.util.ConcurrentModificationException: Document changed during select
     at org.apache.xmlbeans.impl.store.Path$XbeanPathEngine.next(Path.java:459)
     at org.apache.xmlbeans.impl.store.Cursor._toSelection(Cursor.java:931)
     at org.apache.xmlbeans.impl.store.Cursor._getSelectionCount(Cursor.java:946)
     at org.apache.xmlbeans.impl.store.Cursor.getSelectionCount(Cursor.java:2706)
     at org.apache.xmlbeans.impl.values.XmlObjectBase.selectPath(XmlObjectBase.java:468)
     at org.apache.xmlbeans.impl.values.XmlObjectBase.selectPath(XmlObjectBase.java:446)
     at com.eviware.soapui.support.xml.XmlObjectConfigurationReader.readString(XmlObjectConfigurationReader.java:89)
     at com.eviware.soapui.impl.wsdl.teststeps.datasource.FileDataSource.load(SourceFile:304)
     at com.eviware.soapui.impl.wsdl.teststeps.datasource.AbstractDataSource.init(SourceFile:70)
     at com.eviware.soapui.impl.wsdl.teststeps.datasource.AbstractDataSource.init(SourceFile:20)
     at com.eviware.soapui.support.registry.AbstractRegistry.build(AbstractRegistry.java:59)
     at com.eviware.soapui.impl.wsdl.teststeps.WsdlDataSourceTestStep.<init>(SourceFile:103)
     at com.eviware.soapui.impl.wsdl.teststeps.registry.DataSourceStepFactory.buildTestStep(SourceFile:31)
     at com.eviware.soapui.impl.wsdl.testcase.WsdlTestCase.createTestStepFromConfig(WsdlTestCase.java:317)
     at com.eviware.soapui.impl.wsdl.testcase.WsdlTestCase.<init>(WsdlTestCase.java:108)
     at com.eviware.soapui.impl.wsdl.WsdlTestCasePro.<init>(SourceFile:31)
     at com.eviware.soapui.impl.wsdl.WsdlTestSuitePro.buildTestCase(SourceFile:64)
     at com.eviware.loadui.components.soapui.SoapUISamplerComponent$SoapUITestCaseRunner.getTestCase(SoapUISamplerComponent.java:1049)
     at com.eviware.loadui.components.soapui.SoapUISamplerComponent$SoapUITestCaseRunner.run(SoapUISamplerComponent.java:962)
     at com.eviware.loadui.components.soapui.SoapUISamplerComponent.sample(SoapUISamplerComponent.java:831)
     at com.eviware.loadui.impl.component.categories.RunnerBase.doSample(RunnerBase.java:496)
     at com.eviware.loadui.impl.component.categories.RunnerBase.access$700(RunnerBase.java:70)
     at com.eviware.loadui.impl.component.categories.RunnerBase$Worker.run(RunnerBase.java:773)
     at com.eviware.loadui.util.dispatch.CustomThreadPoolExecutor$Worker.run(CustomThreadPoolExecutor.java:188)
     at java.lang.Thread.run(Unknown Source)


    Following is my datasource configuration for all of my test cases -
    • chmelvv's avatar
      chmelvv
      Occasional Contributor
      8 years ago

      May be it is too late to answer, but I had the same error.

       

      It was because I have a lot of included in each other DataSource/DataSourceLoop structures and in one of DataSourceLoop corresponsing DataSource parameters was set to old non existing step.