Hello Manav,
The following documentation article may be useful to you: Load Testing: Checking Whether Server Supports a Certain Number of Users
There is no single configuration to achieve the test objectives you have described. In general, you will want to use a Stepwise load profile so that you can see at what point performance issues may occur. A good starting point is to identify what kind of peak load your application currently receives. If such data is not available, you may make an educated guess based on the amount of concurrent load you may need to support.
For example, to find the crash point for my application, I started by running a test for 100 VU, then 500 VU, then 1000 VU and so on. My 100VU test passed with no problems. My 500VU test also passed, but I noted that Page Load Time was much worse. My 1000VU test failed, and the graph is shown below:
I can see that errors occurred, but not until my load had ramped up to 1000VUs. Therefore, I ran another test for 1500VUs:
In this test, I can clearly see that the server began to give me errors around 1100 users. I then consulted the Details report to see exactly what errors occurred. This article on Analyzing Results may also be useful to you.
Hope this helps.
Hi Ryan,
Thank you a lot for such detailed reply!
Could you please extend it with the information about the used hardware? (CPU, memory, OS, disk type (SSD, disk array, ...), maybe motherboard/chipset, etc that is relevant) Was you able to simulate 1.5k VUs on the single system or used distributed system?
This information can be useful to be able to make an initial estimations about hardware requirements and possible expectations for this or that planned load test.
Hi Alex,
The process I outlined in my post was intended to be a guide for how you might approach stress testing. My own test conditions are unrealistic, as I used LoadComplete to test a locally hosted application, so network was not a consideration. As such, I was generating the load and hosting the app on the same machine (a laptop — Intel i7-3540M 3.0GHz, 8GB RAM, SSD — with an ASP.NET app hosted on IIS 7.5). I was actually surprised this environment could handle as much load as it did.
Hope this helps.
Hi Ryan,
I never was able to be even close to your load figures...
In my case with a laptop Intel i7-4720HQ 3.5GHz, 8GB RAM, processor gets 85-100% load for 7-9 virtual users.
> I used LoadComplete to test a locally hosted application
I am not sure that this is too bad as I still don't know how to measure network delay without the tools like AlertSite or some special setup that makes it possible to measure the time spent in the network. (I will appreciate any piece of advice or experience in this area.) So, I consider load testing within the well configured local network not a bad approach to minimize the impact of the uncnown network delay time.
Thanks for the valuable feedback.But now my question is
1)"If is test for 150 Vusers,test is coming as passed but again running it twice or many times times then error arrives at same 150 VU users.So how can I judge that 150 user are handled by the site or not because sometimes 60% errors are appearing sometime 0 at the same 150 Vusers.
One more question is that:
2)Some time at 200 vusers, no errors are arrived but sometime at 150 users errors arrived,then what will I consider?What is the point in giving error in 150 vuser but not in 200 vusers.
3)And to check the crash point, should I add some other parameters(time/duration of test,loop count etc.) or only by putting virtual users,we'll get to know the crash point.
4)And last,say if on 1000 vusers no error is coming,then on the basis of this test,we can take a decision that it can handle more than 1000 users?Is this test can be solely trustworthy that it can easily handle 1000 users at a time.?
Hi,
I hope that Ryan will correct me where I am wrong and add his knowledge, but here is my opinion:
-- Load tests are not deterministic but more statistical ones. This is because they are driving the complex system that consists from several components each of which is controlled separately and independently (e.g. data server is controlled by its own runtime engine that is absolutely not aware about web server engine and vise versa and thus those two engines do not coordinate their activities) and that communicate via commutated lines. The above means that the fact that the tests passed once for 100 VUs does not guarantee that it will definitely pass for another time. Instead, you should say something like: "With the probability of 95% 100 VUs will be able to browse the list of our products without any problem. For the remained 5%, not more than 10 VUs out of 100 (i.e. 10%) might not get the image of this or that product from the first attempt and not more that 2 VUs out of 100 might not get a list of products at all";
-- If the deviation of the number of failed VUs is small and the number of failed VUs itself is not big, then you may consider that the current number of VUs is the current limit for your system. I.e. for the system as a whole, which means that, for example, the bottleneck might be not in the web server, but in the communication channel between web server and the database;
-- If the deviation of the number of failed VUs is big for the same number of VUs (i.e. 2 VUs out 100 fail in one test run but in another test run 20 VUs fail and so on), you must examine the whole tested system and find out what component causes this or that failure. Here you might require the help from developers, network guys, etc. because the failures might be caused not only by the problems in your developed code, but, for example, the garbage collector service might start and cause the delays that lead to failures. So, without the knowledge of what causes some VUs to fail, it is not possible to answer whether or not the given number of VUs be served by your tested system and why, as per your example, test for 150 VUs may fail while the same test for 200 VUs will pass.
