Forum Discussion

bruinsr's avatar
bruinsr
New Contributor
13 years ago

generating load versus processing time

Hi All,

I am using loadUI 1.0.1 and soapUI 3.6.1 for performance testing our system.
We have two different requirements that our system must satisfy. One is that our system has to deal with 14 concurrent users in a normal case scenario and 140 concurrent users in an extreme case scenario. I used the fixed load generator in achieving that and it works. However using the fixed load generator does not guarantee the second requirement, namely that the system has to process 14 requests (read transactions) per second in the normal case scenario and 140 requests per second in the extreme case scenario. I know that there is some overhead to consider but which loadUI components do I need to use to test that our system has processed 14 requests per second and how? Is this possible in loadUI?

Best regards,

Ronald

4 Replies

  • AndyHughes's avatar
    AndyHughes
    Regular Contributor
    Hi Ronald,

    LoadUI can do this easily (and much more besides). But in my opinion you need to properly understand what exactly you mean by 140 requests per second. Your test will presumably be run for a certain length of time, during which there will be numerous 'seconds'. (It's not really feasible to run a test for one second.). So you need to look at the stats generated in the table output which is aggregated data for the last 'x' seconds. (I'm not sure if you can get the actual data out for each individual request).
    Do you mean 140 complete in 1 second? Under what circumstances? You can send all 140 straight away if you like using the 'burst' functionality on the fixed rate generator. Or just use the same generator to create 140/sec/min whatever. Guess it depends on how your system performs and how long typically these requests take. If they take almost a second each then you will need to have them running concurrently otherwise it will take 140 seconds to complete. (But thats with a fixed load of 1 - which is daft I know). But if they take in the order of milliseconds to complete then a fixed load of 1 isn't so stupid as it will actually complete 140 requests within your allotted second.

    Also fixed rate isn't that much different from concurrent users either. (What are those users for, if not to create a situation where there are lots of requests generated per second?). So I'm not really sure how your test number 2 is any different from test 1. It's all just about load on the system.
    Presumably the test with 14 per second is supposed to be run for some length of time to simulate normal usage whereas the 140/second will be run for a short time to simulate a spike in usage?
  • bruinsr's avatar
    bruinsr
    New Contributor
    Hi Andy,

    Thanks for your reply. I am not sure if I understand correctly but let me ask you this.
    If our system should be able to process 14 transactions per second in a normal situation and 140 transactions per second in an extreme situation (as you said, to simulate a spike in usage) what components in loadUI should you use to test this.
    With process I mean processing in the database. Our application logs the time of the incoming message and the time of the outgoing message and the difference between those times is the processing time that I talk about.

    Currently I use the fixed load generator to simulate 14 concurrent users, but the result is that the application only processes about 2 requests per second in the database and not the required 14 per second. I feel that this approach is not the right way to go.

    Any information is appreciated. I am just learning.

    Ronald
  • AndyHughes's avatar
    AndyHughes
    Regular Contributor
    Hi Ronald,

    That may well be the case. So if your fixed load generator is maintaining a load of 14 (you can see if it is in the runner component. Sometimes even though you have selected a fixed load of 14 you may not be running 14 concurrently, this would depend on the interval you have selected in the fixed load generator. If this is not low enough, it may not be 'releasing' the requests at a high enough rate to ever get to 14 concurrent requests.) But if it is, and you are only processing 2 per second then it may be that your service is just slower than you would like and only 2 complete per second.

    I'd be looking at the average time output on a table log and multiplying that by 14 to see if it is greater than 1 or multiply by 140 in the second scenario.
    The actual set-up should be straight forward and I don't think you are doing anything wrong. Generator, Runner, stats module, table log, I think that's all you need.

    Andy
  • SmartBear_Suppo's avatar
    SmartBear_Suppo
    SmartBear Alumni (Retired)
    Hi,

    To generate 14 requests per second, or 140 requests per second, you should use a Fixed Rate Generator (set to either 14 or 140 requests per second). This will use as many Threads as required (without exceeding the max number of threads in the configuration) to send the desired number of requests. It's quite possible that this won't get you 14 or 140 requests processed per second, but this would be either due to your client not being able to generate enough load (not very likely, as both of these values are relatively low), or that your server isn't able to process the requests fast enough (much more likely, depending on how much work the server has to do per request). If this is the case, you should be able to see this by looking at the different numbers of the Runner: The Running count will likely max out and stay at the max number of concurent requests (100 by default), and the Queued counter will start to increase. Once the Queued counter reaches its max, requests will start being Discarded. If this is happening, it means that the server isn't able to respond to all the requests, and increasing the rate, or the allowed number of Threads will likely only worsen the performance of the server.

    Regards,
    Dain
    eviware.com