Do iterations running in real time take longer than "Gross Duration" + "Delay between Iterations?"

[zachglatt]

I have a test that has 2500 users running a URL navigation test over 11 minutes (1 minute ramp-up, 10 minutes peak).

The average Gross Duration was 10.32 seconds and the Delay Between Iterations is set to 4 seconds. Using the 10 minute test duration, I'm seeing way fewer tests ran than my math suggests they SHOULD have. My goal is to find out how many total times I hit the URL over the course of the test run.

The math suggests...

10 minutes is 600 seconds. I divide that into the sum of the Gross Duration and Delay setting.

600 / 14.32 = 41.9  theoretical iterations by each user.

41.9 x 2500 virtual users = 104,748 theoretical executions of the test (104,748 URL hits).

However, the Statistics tab in my LoadNinja test run says only 45,330 total script steps were run (it's a one-step script). My error rate was 1.03% so that shouldn't play into anything too much.

What is causing this massive difference between the total iterations I expected and the total iterations it actually ran? Does the cycle of each iteration from each user generally take longer than these 14.32 seconds I expected? Maybe LoadNinja takes longer because of processing/setup/etc and I should just look at the completed script count itself?

[Tom_S]

Hello zachglatt,

It is hard to say without access to the load test result what could be causing the lower than expected amount of iterations.

The math you did should be more or less accurate, maybe not exactly the number but it should be something around that. To identify the possible issue with an unexpected, lower amount of steps, please create the support request providing us with the URL to the load test, so we can investigate the possible issues. Based on the load test I will be also able to confirm with our team the possible issues in the number of iterations executed. 

https://support.smartbear.com/loadninja/message/

Greetings,

Tom

[TimeCop1983]

Hey, I just wanted to chime in and say that I completely agree with Tom_S's suggestion - it sounds like the best route to take. That being said, I wanted to provide some additional information about iteration tests. Just so you're aware, iteration tests do tend to take longer to run than regular tests, as there are additional setup, start, and stop iteration phases involved. So it's good to keep that in mind when planning your testing strategy.