Forum Discussion
Hi,
The following is my current understanding. Note, not all bullets were verified in a real-world life, might appear to be incorrect (at some points) and I will appreciate comments / corrections from others.
-- Initial assumption is that user actions (read - scenarios) do not depend on on the previous actions of this user. For example, if the user of the tested automotive trading web application looked at Ferrari, it is out of our scope to deal somehow with the fact that he/she at the application's sidebar will be suggested also to take a look at brand new Lamborgini as well. Things like that should be verified via functional tests and in our scenario we will just know that one or several requests for the sidebar are executed along with the main request(s);
-- Recorded scenarios must not be long and complex. The reason for this is that as long as the given web page evolves, the number of requests that it generates as well as the target of these requests may be changed by developers. So it should be easy for you to re-record the changed traffic;
-- You may craft more complex scenarios by creating a new blank scenario and populating it with the already existing scenarios using the Call Scenario operation;
-- With all required scenarios ready, consider their distribution in a real world using logs from your web server. E.g. 60% of visitors just browse for one car, 35% browse for two cars; 2% additionally look for the list of distributors, 0.2% do a purchase, etc. Based on this distribution, consider the number of virtual user groups, connection speed and browser for each group, the number of users in each group (http://blog.smartbear.com/performance/how-to-determine-your-virtual-user-quota/, https://smartbear.com/learn/performance-testing/load-testing-vu-calculator/ and https://www.google.com/?q=load+testing+required+number+of+virtual+users), what scenarios will be executed by this or that group and whether or not the test is worth to be distributed on several machines;
-- Create Tests using the results of the above considerations.
Perhaps it might help if I provided some detail of our application. It is a web based software package for the administrative side of K-12 schools...ranging from student and family demographics information, student data (attendance, schedules, health, discipline, standardized test results, etc), teacher gradebooks (and attendance taking by teachers), family and student portal to the student information. So, during the course of a day, the list of activities occurring that could reasonable need scenarios built include:
- Enrolling/registering students and updating data - 200-300 users
- Maintaining and creating student class schedules - 200-300 users
- Discipline offense and response entry - 200-300 users
- Viewing student data in the administrative portal - 300-500 users
- Accessing/updating historical data such as grades, enrollments (db reads) - 200-300
- Teacher entry of absences in their class - 2000 (not all concurrently in groups, such as all H.S. teachers)
- Teacher creation of grading assignments and updating of assignments - 2000 users throughout day
- Reports of data - 100-300 users
- Viewing of student data with some update options by parents - 40,000 potentially over course of day
- Viewing of student data with some update options by parents - 75,000 potentially over course of day
With that type of real world activity,
1. While we can generate scenarios and parameterize entries, is it realistic to run such a test continually over an 8 hour day cycle?
2. With at least 10 parameterized scenarios running continuously, will monitoring of the activity as well as interpretation of the results become too massive...other than to determine whether such activity reaches the max stress point.
Does that help?...
- AlexKaras9 years agoChampion Level 3
Hi,
> 1. While we can generate scenarios and parameterize entries, is it realistic to run such a test continually over an 8 hour day cycle?
I don't expect any problem with this so far. https://support.smartbear.com/viewarticle/78727/ and https://support.smartbear.com/viewarticle/78821/.
> 2. With at least 10 parameterized scenarios running continuously, will monitoring of the activity as well as interpretation of the results become too massive...other than to determine whether such activity reaches the max stress point.
Client-side monitoring is described here: https://support.smartbear.com/viewarticle/78788/ and the server-side is described here: https://support.smartbear.com/viewarticle/78462/. Results analysis is described here: https://support.smartbear.com/viewarticle/78920/.
The complexity of analysis depends on your application. Basically, if you run all your load tests within planned time interval and none of the tests fail, server replies were within acceptable time interval and servers monitoring do not show any unwanted trends (like memory consumption increase or disk access time problems) then you may feel (more or less;) ) confident.
Though if the problems are noted, then you will have to figure-out their root and this might require your cooperation with developers, database admins, etc.
Related Content
- 4 years ago
Recent Discussions
- 3 years ago
- 3 years ago
- 4 years ago