Inconsistent Load Test Results among SOAPui Versions
Hello,
We are load testing our Web application with both SOAPui version 4.5 and JMeter, and using the same SOAP script and load test scenario on both applications. The load test is burst, with 20 virtual users running for 180 seconds and no burst delay. JMeter processed 1588 requests in 180 seconds and SOAPui 4.5 processed 482. This is a dramatic difference in load test performance results. The one thing that was noticed with JMeter is how it continuously sent requests to the server to the point where it filled up all of the available threads, resulting in requests waiting in the queue to be processed. When SOAP version 4.5 was testing, there were many less requests getting sent to the server for processing and no requests were waiting in the queue.
A co-worker with SOAPui version 3.6 tested the same scenario, and his results were consistent with JMeter's with 1521 requests processed in 180 seconds. This version also sent requests to the server like JMeter at a continuous loop rate, resulting in requests waiting in the queue to be processed.
These results had me wondering if there are significant differences in SOAPui releases. As a result, I installed SOAPui version 4.0. Version 4.0 about 55% more requests than version 4.5, but was still much lower than when testing with version 3.6. It processed 1026 requests in 180 seconds.
I had been using SOAPui for some time and always wondered why my results would be inconsistent with JMeter and my co-worker who was using SOAPui version 3.6. It appears that version 4.0 and 4.5 have defects in the way that it bursts virtual user requests. Why are the different versions of SOAPui producing dramatically different results?
We are load testing our Web application with both SOAPui version 4.5 and JMeter, and using the same SOAP script and load test scenario on both applications. The load test is burst, with 20 virtual users running for 180 seconds and no burst delay. JMeter processed 1588 requests in 180 seconds and SOAPui 4.5 processed 482. This is a dramatic difference in load test performance results. The one thing that was noticed with JMeter is how it continuously sent requests to the server to the point where it filled up all of the available threads, resulting in requests waiting in the queue to be processed. When SOAP version 4.5 was testing, there were many less requests getting sent to the server for processing and no requests were waiting in the queue.
A co-worker with SOAPui version 3.6 tested the same scenario, and his results were consistent with JMeter's with 1521 requests processed in 180 seconds. This version also sent requests to the server like JMeter at a continuous loop rate, resulting in requests waiting in the queue to be processed.
These results had me wondering if there are significant differences in SOAPui releases. As a result, I installed SOAPui version 4.0. Version 4.0 about 55% more requests than version 4.5, but was still much lower than when testing with version 3.6. It processed 1026 requests in 180 seconds.
I had been using SOAPui for some time and always wondered why my results would be inconsistent with JMeter and my co-worker who was using SOAPui version 3.6. It appears that version 4.0 and 4.5 have defects in the way that it bursts virtual user requests. Why are the different versions of SOAPui producing dramatically different results?
