LoadTest Response Time Difference between SoapUI, HP LoadRunner and Apache JMeter
Hello everybody,
I am currently doing L&P testing using LoadRunner ( WebHTTP Protocol ). The test setup:
1 Virtual User / Thread
1 SOAP Call every Second
1200 SOAP Calls during the test.
Data is the same among all SOAP Requests
To test single request i usually use SoapUI as most of our developers do. During my tests i saw huge differences among various LoadTest Tools. Here's a small Table to illustrate the differences I saw:
Tool | Min | Avg | Max | 90%
------------------------------------------------------------
LoadRunner | 0.080 | 0.121 | 0.262 | 0.144
SoapUI LoadTest | 0.015 | 0.025 | 0.140 | n/a
JMeter | 0.056 | 0.064 | 0.175 | 0.067
I am wondering, why the response times from the SoapUI LoadTest are so much faster than the ones from JMeter and especially from LoadRunner. Are there differences how they are calculated? I have a problem to Justify my Measurements with LoadRunner when LoadRunner has most of the reported response times above 100 ms (without heavy load ) and the developer sees response times way below 100 ms for most of the SOAP requests.
Thank you for your help!