[Resolved]Response time differences between soapUI 2.5 beta1 and soapUI 3.0.1

Hi Sean,

that does look very odd.. which of these results seem more "real" to you? ie if you send standalone request to your service, are you getting short or long response times?

I'll try to reproduce with a local mockservice/loadtest to see if I get the same discrepancy, I'll let you know.

regards,

/Ole
eviware.com

Hi Ole,

Running a function test with the same base script in both version of soapUI I get:

-- in soapUI 2.5 beta1, execution time = 982ms
-- in soapUI 3.0.1, execution time = 4,125ms

Interestingly enough, I then tried replicating the same calls using our application.  I ran Fiddler2 to inspect the timings of the resulting web service calls.  I measured each discrete request/response to the server and added up the times.  I got 542ms, which is closer to the soapUI 2.5 beta 1 results, which I feel are closer to the actual experience through our application.

Hi!

ok, thanks.. do you get the when doing repeated requests? (The first request will usually take longer, although it shouldn't affect the value.. hm.. )

Let me see what I find out..

regards,

/Ole
eviware.com

Hi again Ole,

The results below are for repeated runs of the same functional test script.

soapUI 2.5 beta1 (ms)
#1: 2,351
#2: 189
#3: 169
#4: 171
#5: 161
#6: 203
#7: 163
#8: 191
#9: 194
#10: 188

soapUI 3.0.1 (ms)
#1: 5,142
#2: 1,258
#3: 1,512
#4: 1,554
#5: 2,663
#6: 1,506
#7: 1,200
#8: 1,626
#9: 1,199
#10: 1,617

The results are consistent and still appear to point to significant differences between the soapUI versions.

Hi - I haven't seen any feedback on this issue and it remains unresolved.  Could you please update me on your investigations?  Many thanks.

Hi Sean,

we are still investigating and trying to reproduce.. I'll let you know as soon as we have something found out..

regards!

/Ole
eviware.com

Hi Ole - thanks for that.

Sean

Hello,

I created simple project and logged duration of each test case during load test, but I was not able to reproduce this problem. Times are similar. Check attachment. Could you provide us with more information? Or check a project which I used to reproduce this a problem and add what were you doing so we can reproduce it and fix it.

thanks,
robert

Hello Robert,

I ran load tests on both the tests included in the sample project file you sent me.  I ran these in both versions 2.5 beta 1 and 3.0.1.  Like you, I get similar results, in the order of 140ms.  I then tried cloning the test steps to give multiple steps in each test on a hunch that perhaps chained test steps might be a causal factor.  This wasn't the case and the timings were again similar.  I also tried including  multiple 209.85.129.104 calls (with different values) within the < getgeoip=""> method.  Once again, the timings were similar.  I went back to  my original test script and ran it again in both versions.  I still see the significant timing difference.  Delving deeper into the individual test steps in my script I see that one web request in particular is contrubuting to the timing difference.  I am going to send you a private message with a copy of the 'offending' web request.  I am wondering if the fact that it contains 100 separate groovy scripts might be a factor.  Just a thought.  Perhaps you could review this call and give me your thoughts?

Thanks
Sean

Update:  I think it is related to the execution of Groovy script in v.3.0.1.

I cloned the offending web request and replaced the 100 instances of groovy script with hard-coded values.  I then disabled the original web request and enabled the clone.  When I ran this script in 2.5 beta 1 and 3.0.1 I get similar average times in the load test (c.300ms).  When I revert to the oiginal web request I then get c.300ms in 2.5 beta 1 but nearer 8,000ms in 3.0.1.  I think this may be related to the way 3.0.1 executes groovy script on the fly during a script execution - has there been a change here since 2.5 beta1?

Thanks
Sean