Forum Discussion

sarya's avatar
sarya
Frequent Contributor
14 years ago

Analysis of load test results

Hi,



I ran one test for a single user.The load test log gives me result as:time as 323.71 s and connections count as 239. I have to analyse the load test results but the results are somewhat difficult to interpret from the logs. I am attaching the corresponding log.



1. If I calculate the Total time from the column Time(s) of connections ,it comes out to be 352.344.Is it not the same time as the total time displayed(323.71) in the Users part of the load test result.Also how do I evaluate my results .

2. How is the performance(kb/s) calculated for each user.Which bytes(bytes received or bytes sent ) and time value and from which column of the load test results are being used to evaluate the perfromance in kb/s .

3. Also the start delay column of the load result in connections part shows some delay for each of the connections. What does it depict? I did not mention any start delay while my load test run.

4. How are minimum and maximum response time calcualted for all the connections and which values from the log to be used for calculation ?

10 Replies

Sort By
  • sarya's avatar
    sarya
    Frequent Contributor
    14 years ago
    Hello,



    I have one more question. How to measure the average response time for the total task execution.

    I calculated average time to process a request by dividing Total request exceution summary by total time.

    What are the parameters needed to calculate the average response time as in the load test log,the response time columns are divided into minimum and maximum response times.So which values to take into account to calculate the total response time and does it need to be divided by the total time for all the connections or the total time for the task execution ?



    thanks,

    Sumedha
  • AlexeyKolosov's avatar
    AlexeyKolosov
    Staff
    14 years ago

    Hello Sumedha,




    1. If I calculate the Total time from the column Time(s) of connections ,it comes out to be 352.344.Is it not the same time as the total time displayed(323.71) in the Users part of the load test result.Also how do I evaluate my results .



    To learn what is included in every time value, please see the HTTP Load Testing - Timing Task Execution help topic.




    2. How is the performance(kb/s) calculated for each user.Which bytes(bytes received or bytes sent ) and time value and from which column of the load test results are being used to evaluate the perfromance in kb/s .



    Let me quote the Load Testing Log Page help topic:



    Values in this column are calculated by the following formula:

    Performance = (Total Bytes Sent + Total Bytes Received) / Time

    These values indicate the server performance when it was working with the connection.




    3. Also the start delay column of the load result in connections part shows some delay for each of the connections. What does it depict? I did not mention any start delay while my load test run.



    The "Load Testing Log Page" help topic describes this value:

    The number of seconds passed from the start of the load test till the moment when the connection was established.




    4. How are minimum and maximum response time calcualted for all the connections and which values from the log to be used for calculation ?



    This can be done for a specific user in the following way:

    - in the Load Testing Log, right-click within the Connections list;

    - select "Show Summary" from the context menu - you will see the Summary panel at the bottom of the connections list;

    - right-click within the Summary panel under the "Min response time" or "Max response time" column and select the type of the summary value you want to see.

    By using this approach, you can see the minimum and maximum response time across connections.




    How to measure the average response time for the total task execution.



    TestComplete does not calculate this information automatically. The only way to get it is to access the Load Testing results from script and iterate through all requests' data to get the average response time. To learn how to do this, please see the Scripting Access to the Test Log Contents help topic.

  • sarya's avatar
    sarya
    Frequent Contributor
    14 years ago
    Hi Allen,



    Thanks for the detailed response. I was able to understand everything from the log except the last few lines from your reply .



    'How to measure the average response time for the total task execution.'

    TestComplete does not calculate this information automatically. The only way to get it is to access the Load Testing results from script and iterate through all requests' data to get the average response time. To learn how to do this, please see the Scripting Access to the Test Log Contents help topic.



    I read the topic contents but still not clear how to retrieve the average response time .Can I sum up all the individual response time and divide by the number of the runs that it took to complete one load test ? Can you please help me with how to iterate through data to get the average time as I am runnign my load test visually and not through the script.

    For example- If I run 10 concurrent users in one task ,each user will have its own run results.So I sum up the Time for all the 10 users and divide by the number of users.Is this the right approach?



    Thanks,

    Sumedha
  • sarya's avatar
    sarya
    Frequent Contributor
    14 years ago
    Hi Allen,



    Thanks for the detailed response. I was able to understand everything from the log except the last few lines from your reply .



    'How to measure the average response time for the total task execution.'

    TestComplete does not calculate this information automatically. The only way to get it is to access the Load Testing results from script and iterate through all requests' data to get the average response time. To learn how to do this, please see the Scripting Access to the Test Log Contents help topic.



    I read the topic contents but still not clear how to retrieve the average response time .Can I sum up all the individual response time and divide by the number of the runs that it took to complete one load test ? Can you please help me with how to iterate through data to get the average time as I am runnign my load test visually and not through the script.

    For example- If I run 10 concurrent users in one task ,each user will have its own run results.So I sum up the Time for all the 10 users and divide by the number of users.Is this the right approach?



    Thanks,

    Sumedha
  • AlexanderM's avatar
    AlexanderM
    Staff
    14 years ago

    Hello Sumedha,





    An example of parsing Load Testing results programmatically can be found in this thread.





    To get the average response time for all 10 users, you need to iterate through all requests of each user and collect the sum of the response time for them all in all connections of all 10 users. The result needs to be divided by the total number of requests.
  • sarya's avatar
    sarya
    Frequent Contributor
    14 years ago
    Hey Alex,



    On the load test anaylsis page, for my load test I ran a test for 10 concurrent users and the average process time is 2172.35 and the user count is 10. So to get total time ,I multiply both of them and it comes as 21723.5 .If I ran the same scenario 10 times,I do the same thing 10 times ,then add up all the total process times and divide it by 100 as the task (using 10 users) ran 10 times.Is this also a correct way to get the average process time for a task having 10 concurrent users that has been run for 10 times.



    Thanks,

    Sumedha
  • AlexeyKolosov's avatar
    AlexeyKolosov
    Staff
    14 years ago

    Hi Sumedha,


    The average process time is not exactly the same thing as the calculated sum of response times mentioned by Alex. Let me quote the "Load Testing Analysis Page" ( http://www.automatedqa.com/support/viewarticle.aspx?aid=2829 ) help topic:



    Avg Process Time, Average virtual user life time (in seconds) during the test run.


    Suppose each of your virtual users sends 30 requests to the web server. So, you need to calculate the sum of response times for all the requests programmatically and then divide the resulting value by the total number of requests (30 requests * 10 users = 300).

  • sarya's avatar
    sarya
    Frequent Contributor
    14 years ago
    Hey Allen,

    According to your reply -


    Suppose each of your virtual users sends 30 requests to the web server. So, you need to calculate the sum of response times for all the requests programmatically and then divide the resulting value by the total number of requests (30 requests * 10 users = 300).





    I have attached the log .In that for the first item in the load testing log for 10 users.I calculated the total response time for the values from the column "Request exceution summary" for each of the 10 users and then sum up the values as 23517.891 seconds .

    The request count for each of the 10 users is "1158",

    So to get the average ,I divided 23517.891 by the total request count for 10 users (11580) .Then the resut came out to be a very very small value.Is this the right way? The average value should be something around 2000s for each user .



    Thanks,

    Sumedha

  • AlexeyKolosov's avatar
    AlexeyKolosov
    Staff
    14 years ago

    Hi Sumedha,


    You got the correct result, I suppose. 23517 seconds / 11580 requests = 2 seconds per request. This looks like the correct response time value. And by the way, 2 seconds * 1158 requests = 2316 seconds per each virtual user.

  • sarya's avatar
    sarya
    Frequent Contributor
    14 years ago
    Hi Allen,



    Thanks so much for the reply.I understood the way to calculate response time. So this means that the Time(s) column in the Users panel shows the times that include test engine initialization time etc  that is the total time includes the times from other operations also.Thats why the value is bigger than that from the 'Request Exceution summary ' column.



    Thanks,

    Sumedha