LoadUI Limits
Hi I've finally got our services hosted on some beefy hardware and started load testing. Everything was going great and I was sending in some pretty hefty volumes (I'm talking in excess of 500 req/...
Forum Discussion
SmartBear_Suppo
Alumni
AlumniHi,
Since I don't know exactly what is happening on your server, this is mainly based on my own speculation. I may be wrong about what is actually happening here, but this seems likely:
The response time increasing causes more and more requests to start queuing up, which may exhaust the number of available concurrent requests. This number is determined by a few different settings in loadUI, and are there for efficiency reasons. Each Runner component has a Max Concurrent Requests setting (in the Basic tab of the settings dialog), which defaults to 100. Generally, when a Runner reaches this limit, it is because the requests are being queued on the server, and sending more requests won't increase the throughput. Instead, the Runner will start queuing requests internally, increasing the Queued counter (as soon as the number of active, or "Running" requests drops below the max, queued requests will be sent out) until it reaches its max (also a setting in the Basic tab). Once this happends the Runner will start discarding requests, which either will or will not be counted as failures depending on yet another setting (Count Discarded Requests as Failed, false by default).
So, what may be happening when you see greatly increased response times is: The server is queueing requests, because it is at full capacity. This causes the number of active connections to reach the maximum, causing the Runner to start internally queuing. The queue quickly fills up, and requests are discarded, leading to a lower overall throughput.
Aside from the Runner settings, there are two global setting which limits the global thread pool used in loadUI (available in the Workspace settings dialog, under Execution), Max internal threads, and Max internal thread queue size. These settings control the global thread pool used by loadUI, and is shared among all Components and several other parts of the application. This sets a hard limit on the total number of concurrent requests among all Runners.
It may seem a bit limiting that these limits are in place, but the reality is that increasing the request rate once the server has started queuing requests will almost certainly lower the throughout instead of increase it, as the server it already at capacity (hence the queuing in the first place).
Regards,
Dain
SmartBear Software
Since I don't know exactly what is happening on your server, this is mainly based on my own speculation. I may be wrong about what is actually happening here, but this seems likely:
The response time increasing causes more and more requests to start queuing up, which may exhaust the number of available concurrent requests. This number is determined by a few different settings in loadUI, and are there for efficiency reasons. Each Runner component has a Max Concurrent Requests setting (in the Basic tab of the settings dialog), which defaults to 100. Generally, when a Runner reaches this limit, it is because the requests are being queued on the server, and sending more requests won't increase the throughput. Instead, the Runner will start queuing requests internally, increasing the Queued counter (as soon as the number of active, or "Running" requests drops below the max, queued requests will be sent out) until it reaches its max (also a setting in the Basic tab). Once this happends the Runner will start discarding requests, which either will or will not be counted as failures depending on yet another setting (Count Discarded Requests as Failed, false by default).
So, what may be happening when you see greatly increased response times is: The server is queueing requests, because it is at full capacity. This causes the number of active connections to reach the maximum, causing the Runner to start internally queuing. The queue quickly fills up, and requests are discarded, leading to a lower overall throughput.
Aside from the Runner settings, there are two global setting which limits the global thread pool used in loadUI (available in the Workspace settings dialog, under Execution), Max internal threads, and Max internal thread queue size. These settings control the global thread pool used by loadUI, and is shared among all Components and several other parts of the application. This sets a hard limit on the total number of concurrent requests among all Runners.
It may seem a bit limiting that these limits are in place, but the reality is that increasing the request rate once the server has started queuing requests will almost certainly lower the throughout instead of increase it, as the server it already at capacity (hence the queuing in the first place).
Regards,
Dain
SmartBear Software