Hi Ryan,
Thank you a lot for such detailed reply!
Could you please extend it with the information about the used hardware? (CPU, memory, OS, disk type (SSD, disk array, ...), maybe motherboard/chipset, etc that is relevant) Was you able to simulate 1.5k VUs on the single system or used distributed system?
This information can be useful to be able to make an initial estimations about hardware requirements and possible expectations for this or that planned load test.
Hi Alex,
The process I outlined in my post was intended to be a guide for how you might approach stress testing. My own test conditions are unrealistic, as I used LoadComplete to test a locally hosted application, so network was not a consideration. As such, I was generating the load and hosting the app on the same machine (a laptop — Intel i7-3540M 3.0GHz, 8GB RAM, SSD — with an ASP.NET app hosted on IIS 7.5). I was actually surprised this environment could handle as much load as it did.
Hope this helps.
Hi Ryan,
I never was able to be even close to your load figures...
In my case with a laptop Intel i7-4720HQ 3.5GHz, 8GB RAM, processor gets 85-100% load for 7-9 virtual users.
> I used LoadComplete to test a locally hosted application
I am not sure that this is too bad as I still don't know how to measure network delay without the tools like AlertSite or some special setup that makes it possible to measure the time spent in the network. (I will appreciate any piece of advice or experience in this area.) So, I consider load testing within the well configured local network not a bad approach to minimize the impact of the uncnown network delay time.
Hi AlexKaras
When LoadComplete — or more specifically, the Remote Agent Service that is used to generate load — is running a test, it will consume all available CPU resources. (As a side-note, we are looking to change this behavior in the future to be more responsible with CPU resources.) Monitoring the CPU on the machine that is generating the load will not provide a useful metric. To determine load capacity on a local machine, I would be looking to see how the web server responds to all the requests that are generated. In my example above, I determined load capacity by looking at server responses. When the server was overloaded, I started to see HTTP status codes of 500 Internal Server Error.
Does this help?
Hi Ryan,
> Does this help?
Oh, YES, it does!
Thank you a lot for this information.
Indeed, I was able to execute my test for 16 VUs without an error. But for 40 and 50 VUs I started to get login failures for some users and several connections were closed with the socket error code #5 (connection closed). There were no 5xx http status codes returned though.
Unfortunately, I don't have access to the test server, but my current understanding is that the socket error #5 is a kind of equivalent of the 5xx http status code. Is that correct?
P.S.
> [...] the Remote Agent Service [...] consume all available CPU resources.
Do you know if there is any indication that there is not enough CPU resources on the computer running the load test?
Or 2000 VUs mentioned in documentation must be reliably simulated on an average modern laptop / desktop?
