Not able to generate load properly

Elvorin wrote:
1. I've set a test case with fixed rate generator. It's supposed to generate 25/sec, with burst size 25 (tried 1 too). But it doesnt send that many requests per second. 3. Requests/sec graph shows it going down from 25 to 1 or 2 within few seconds.

The Fixed Rate Generator is almost certainly generating 25/sec if that's what you set it to. You can check this by connecting the generator directly to a Table Log.

However, it seems that your soapUI Runner starts sending requests as it gets trigger messages from the Generator -- but since the requests doesn't complete very fast (maybe
because of the ConcurrentModificationException?), the runner starts queuing trigger messages up and eventually also discarding them. These mechanisms are explained here in the documentation.

Elvorin wrote:
2. It starts throwing error pretty soon saying 'java.util.ConcurrentModificationException: Document changed during select'. I have property transfer from the response in my test case, for logging purposes. That is causing the failure (confirmed by enabling/disabling that step).

The question is to where you transfer the property. Each virtual user (or trigger message) in a soapUI Runner gets its own copy of the soapUI TestCase -- but not of the entire TestSuite or Project (that would be way to heavy for any computer to run). Thus, TestSuite and Project properties are shared between Virtual Users, and will cause concurrent issues if you don't avoid writing to them concurrently. Read more about this here.

Hope this helps!

Henrik
SmartBear Software

The properties are test case properties. One transfer is done before making the SOAP call to capture the input and one transfer after the SOAP call to get a value from response. So it's like

DataSource [Reading from comma separated file]
Property Transfer [to a property of this test case to capture the read value]
SOAP call
Property Transfer [to a property of this test case to capture a value within response]

Elvorin wrote:

The properties are test case properties. One transfer is done before making the SOAP call to capture the input and one transfer after the SOAP call to get a value from response. So it's like DataSource [Reading from comma separated file] Property Transfer [to a property of this test case to capture the read value] SOAP call Property Transfer [to a property of this test case to capture a value within response]

Hmm... is the DataSource set to shared in its settings?

/Henrik

Yes.

Just FYI, I'm not seeing the issue anymore... not sure how it went away.

Nope. I'm seeing lots of java.lang.Exception: Failed to prepare testStep [DataSource]; java.lang.Exception: DataSource not set in step [DataSource]

It's not that the particular call is failing 100% of the time, it's intermittently.

Also just noticed few of the infamous "java.util.ConcurrentModificationException: Document changed during select";

Hi again!

Could you please send me the project? My email is my first and last name separated by a dot, at smartbear.com.

Thanks!

Henrik Olsson
SmartBear Software

I'm afraid that will not be possible. I cannot disclose the project. It'll be against company privacy policy. It's a large scale eCommerce platform project. Currently we are trying to test a particular service of it.

If you are looking for any particular information, please let me know, and I'll be glad to provide you that.

To give you an idea how we are setting up our project -

we set up test cases in SOAPUI. Test steps are very straightforward. It has a DataSource to read input from a comma separated file, next it makes a SOAP call for the service we wanna test. SOAP call has 2 assertions, SOAP Response and Not SOAP Fault. For every test case data source is shared and set for looping through during the load testing. Then we setup our LoadUI test case. There we get all these service calls as separate SOAPUI runner and attach these with generators (Fixed/Random) based on our requirement on call distribution. Also we attach the error out to a table log.

Above is our intended setup. But then we noticed on error, no useful information was getting logged to debug, like input/output etc. So I added properties to each of the test case in SOAPUI, added a property transfer step to populate them and then in SOAPUI Runner enabled the test case property logging and attached results output to another table log component. This is our current setup.

Seeing a lot of

"23416","[]","18:54:47:648","java.lang.Exception: Failed to prepare testStep [DataSource]; java.lang.Exception: DataSource not set in step [DataSource]".

And yes, my datasource is setup. If I run with a small load, it does work without any issue.

Can I get a response to the last issue please? I'm seeing a lot of this errors recently. And nothing in my soapui test case has changed.