cancel
Showing results for 
Search instead for 
Did you mean: 

Response time

Highlighted
Contributor

Response time

Hi,

I am using TC HTTP Load Test to stress test my ASP.NET application. I tried to test with 1, 10 and 50 virtual users to login to my application. The testing is run from script, add 50 users to group and run at the same time.





Below is the testing result

1 user - < 1 sec

10 users - EVERY user took 10-12 secs

50 users - EVERY user took 55-70 secs





The time above is the time shown in "Time (s)" column of users panel. I have check response time for each request at request panel, the sum of all request for each user is around the time above as well.





What does it meant? My page doesn't contain any logic, is just redenring HTML to show login.aspx. I didn't see any lock as well.





How TC can help in this case?

Beside, how to get End time? I want to know it to know if my IIS doesn't able to handle so many connections or TC process response only after all request sent.





Please advice. Thank you
 
9 REPLIES 9
Highlighted

RE: Response time

Hi Chan,


Most probably, it means that the number of allowed connections is limited on the server machine. For example, the general limitation for Windows XP is 10 simultaneous connections (see this web page for more information). So, if a non-server OS is installed on the server machine, it is possible that the next 10 virtual users will only start after the previous 10 connections are closed.



Best regards,
Alexey

Did my reply answer your question? Give Kudos or Accept it as a Solution to help others. ⬇️⬇️⬇️
Highlighted
Contributor

RE: Response time

Hi,

Windows Server 2003 Enterprise is installed at the server.

I have read another post, which mentioned I should take the Average Process Rime / # of Users in order to get accurate process time. Which meant according to the result in my previous post, each user only took 1-2 secs.





Is it the way to calculate?

Thank you
 
Highlighted

RE: Response time

Hi Chan,


I recommend that you visit the following web pages - they contain information on analyzing load testing results:

http://www.automatedqa.com/forums/d.cgi?cmd=xover&group=automatedqa.public.testcomplete&related=3326...=

http://www.automatedqa.com/support/viewarticle.aspx?aid=8833



Best regards,
Alexey

Did my reply answer your question? Give Kudos or Accept it as a Solution to help others. ⬇️⬇️⬇️
Highlighted
Contributor

RE: Response time

Hi

I have gone thru the link you sent. However, I still unable to get it. Shall you please direct answer me but not pointing me to somewhere else, so that I can understand better without interrupt you again





1. In my 10 virtual user, Avg Process Time is 16.32. Does it meant ONE user is only taken 1.6s (16.32 / 10 users)?

2. The test contains 4 requests

    0001 - Navigate to Login.aspx (No logic, just UI rendering) | Avg Response time in request panel - 16.x s

    0002 - Simulate login (provided username and password and 'click' login button), navigate to mainPage.aspx upon login successfully  | Avg Response time in request panel - 0.076

    0003 - 'Click' logout link to logout | Avg Response time in request panel - 0.03

    0004 - Return to Login.aspx upon logout successfully. | Avg Response time in request panel - 0.05





If I only want to know the avg time taken for request 0002 for each user, should I use 0.076 / 10  users as well? If so, it seem really fast!!! However, if we tried to simulate this request (only one request) manually, it will take about 1.5s.





Question, how to explain this?





Thank you
 
Highlighted

RE: Response time

Hi Chan,




Avg Process Time is 16.32. Does it meant ONE user is only taken 1.6s (16.32 / 10 users)?




No. Let me explain what the "average" term means using a simple example. Suppose you have two values: 15 seconds and 17 seconds. The average of the two values is (15 + 17) / 2 = 16, where 2 is the total number of items summarized. The characteristic you refer to should be called "total", not "average".




If I only want to know the avg time taken for request 0002 for each user, should I use 0.076 / 10  users as well?




There is no way to calculate the value directly. The only way is to create a custom parser which will retrieve the needed data from load testing logs. I can provide you with a sample demonstrating how you can parse load testing logs programmatically if you need it.



Best regards,
Alexey

Did my reply answer your question? Give Kudos or Accept it as a Solution to help others. ⬇️⬇️⬇️
Highlighted
Contributor

RE: Response time

Hi

Yes, it is appreciated if you can share anyway to parse load test result.





Additionally, we don't know how to explain to customer on all the testing result. 16secs for 10 users is it fast or slow? I know it is very subjective. Then how to justify it?





I found that, other tool like Team system provide a lot of graph for difference purpose. I know that we can customize graph in  TestComplete. However, if beginner like me, I have no idea how to plot them for each performance purpose. It would be good if by default that is full of report ready for me to view it.





Thank you
 
Highlighted

RE: Response time

Hi Chan,


The data you need to view depends on your purposes only, so you need to choose the needed columns yourself. The description of each available column is provided in the "Load Testing Analysis Page" ( http://www.automatedqa.com/support/viewarticle.aspx?aid=2829 ) and "Load Testing Log Page" ( http://www.automatedqa.com/support/viewarticle.aspx?aid=2633 ) help topics. The recommended approach (which includes creating some graphs, by the way) was provided in the newsgroup thread that I have already mentioned before:

http://www.automatedqa.com/forums/d.cgi?cmd=xover&group=automatedqa.public.testcomplete&related=3326...=


Here is a quote from that thread:



As far as I understand, the "extreme load test" you performed is a

stress test and the goal of the test is to check whether the tested

application will fail if many users use it simultaneously. The other

test you need to perform is a load test, and this one tests your

appliaction's performance depending on the number of virtual users (see

the "About Performance, Load, Stress and Scalability Testing of Web

Servers" help topic for more information on different testing types). If

you need to perform a load test, I recommend that you use the following

approach:

1. Create a load test.

2. Set an initial number of virtual users.

3. Launch your test.

4. In the test log, use the Summary bar of the Connections panel to

evaluate sum value for the "Request Execution Summary" column. To do

this, right-click somewhere within the panel and select the "Show

Summary" context menu option, then right-click the bar under the

corresponding column and select the "Sum" context menu item.

5. Repeat step 4 for the "Request Count" column.

6. Divide the first value by the second value. This will give you the

average time needed for your web application to process a request. The

value depends on your server's load and may vary for different virtual

users, however if all users are started simultaneously, the difference

should not be significant.

7. Increase the number of virtual users.

8. Repeat steps 3 - 6 for the new number of virtual users. The results

you get during your tests will allow you to plot a chart indicating how

your web server's performance depends on the number of users who work

with the web application.




it is appreciated if you can share anyway to parse load test result



You can export the load testing log in the XML format, parse the generated XML files and store the results in an Excel workbook. Here is a sample script that parses XML files:



function Test()

{

  var LogFolder = "C:\\Log"; 

  Export(LogFolder);

}

function Export(XMLLogFolder)

{

  xcl = Sys.OleObject("Excel.Application");

  var xmlDoc = Sys.OleObject("Msxml2.DOMDocument"); 

  Workbook = xcl.Workbooks.Add();      

  Worksheet = Workbook.Sheets.Add();

  xmlDoc.load(XMLLogFolder + "\\DefaultLog.xml")  

  ExportDefaultLog(Worksheet, xmlDoc); 

  Worksheet = Workbook.Sheets.Add();

  xmlDoc.load(XMLLogFolder + "\\Users.xml") 

  ExportUsers(Worksheet, xmlDoc);       

  xcl.Quit();       

}

function ExportDefaultLog(Worksheet, XMLDoc)

{  

  Worksheet.Name = "Default Log";

  Worksheet.Cells(1, 1) = "Type";

  Worksheet.Cells(1, 2) = "Message";

  Worksheet.Cells(1, 3) = "Time";

  Worksheet.Cells(1, 4) = "Priority";

  Worksheet.Cells(1, 5) = "Link";

  Worksheet.Cells(1, 6) = "Type Description";

  Worksheet.Cells(1, 7) = "Remarks"; 

  DefaultLogItems = XMLDoc.selectNodes("/DefaultLog/DefaultLogItem");

  for (var i = 0; i < DefaultLogItems.length; i++)

  {

    n = i + 2;   

    item = DefaultLogItems.item(i);   

    Worksheet.Cells(n, 1) = item.selectSingleNode(".//Type").text;

    Worksheet.Cells(n, 2) = item.selectSingleNode(".//Message").text;

    Worksheet.Cells(n, 3) = item.selectSingleNode(".//Time").text;   

    Worksheet.Cells(n, 4) = item.selectSingleNode(".//Priority").text;

    Worksheet.Cells(n, 5) = item.selectSingleNode(".//Link").text;

    Worksheet.Cells(n, 6) = item.selectSingleNode(".//TypeDescription").text;

    Worksheet.Cells(n, 7) = item.selectSingleNode(".//Remarks").text;

  } 

}

function ExportUsers(Worksheet, XMLDoc)



  Worksheet.Name = "Users";

  Worksheet.Cells(1, 1) = "Virtual User";

  Worksheet.Cells(1, 2) = "Task";

  Worksheet.Cells(1, 3) = "Workstation";

  Worksheet.Cells(1, 4) = "Time (s)";

  Worksheet.Cells(1, 5) = "Connection Count";

  Worksheet.Cells(1, 6) = "State";

  Worksheet.Cells(1, 7) = "#";

  Worksheet.Cells(1, 8) = "State Description";

  Worksheet.Cells(1, 9) = "Connections"; 

  Users = XMLDoc.selectNodes("/Users/UsersItem");

  for (var i = 0; i < Users.length; i++)

  {

    n = i + 2;   

    item = Users.item(i);   

    Worksheet.Cells(n, 1) = item.selectSingleNode(".//VirtualUser").text;

    Worksheet.Cells(n, 2) = item.selectSingleNode(".//Task").text;

    Worksheet.Cells(n, 3) = item.selectSingleNode(".//Workstation").text;

    Worksheet.Cells(n, 4) = item.selectSingleNode(".//Times").text;

    Worksheet.Cells(n, 5) = item.selectSingleNode(".//ConnectionCount").text;

    Worksheet.Cells(n, 6) = item.selectSingleNode(".//State").text;

    Worksheet.Cells(n, 7) = item.selectSingleNode(".//N").text;

    Worksheet.Cells(n, 8) = item.selectSingleNode(".//StateDescription").text;

    Worksheet.Cells(n, 9) = item.selectSingleNode(".//Connections").text;

  } 

}




16secs for 10 users is it fast or slow?



It depends on the tested application only. I believe the requirements should be specified by the application developers - they need to know what exactly they want to test, the expected results, the desired behavior of the tested application and the purpose of the load test.



Best regards,
Alexey

Did my reply answer your question? Give Kudos or Accept it as a Solution to help others. ⬇️⬇️⬇️
Highlighted
Contributor

RE: Response time

Hi


Thank you.


I have read the post.






Question, the example shown how to get avg time for each request. If I want to know avg time for each user, does it meant






sum of Avg Process Time / Sum of Users?






>>It depends on the tested application only. I believe the requirements should be specified by the application developers - they need to know what exactly they want to test, the expected results, the desired behavior of the tested application and the purpose of the load test.







Developer expectation is the whole login task (4 request as shown in prior post) is less than 3 seconds. Does it meet? How to determine it?







Thank you
 
Highlighted

RE: Response time

Hi Chan,


You need to calculate the sum of values displayed in the Request Execution Summary column of the Connections panel in the load test log - this will give you the total time for a virtual user. To learn how to do this, please see the "Viewing Summary Values" help topic ( http://www.automatedqa.com/support/viewarticle.aspx?aid=2698 ).



Best regards,
Alexey

Did my reply answer your question? Give Kudos or Accept it as a Solution to help others. ⬇️⬇️⬇️
New Here?
Join us and watch the welcome video:
Announcements
Top Kudoed Authors