Load Tests are not fully executing
[ Spoiler ] This is output from a LoadTest that I just ran. You can see that the cnt is different per TestStep 1 vs 2 vs 3 50 for test step 1, 26 for test step 2, 17 for test step 3 I have de...
Forum Discussion
Maybe I mis-interpretted how the 'Runs per Thread' works? I assumed that each test step would be given 5 threads and each test step would run 10x per thread; equally 50 executions.
Since the 3rd test step is taking longer, it is only executing around 17-20 times. It looks like the LoadTest ends as soon as any one of the test steps reaches the 50 limit.
Is there any way to write the tests so that each of these runs a total of 50x?
Should each test step be it's own test case instead, with its own LoadTest associated with it?
Hi,
Its hard guess exactly what might be wrong without seing your TestCase details. In my experience "runs per thread" should mean exactly what you expected e.g. 5 threads, limit 10, "runs per thread", should result in the TestCase being run a total of 50 times, 10 times per thread. However, there are situations where things can get somewhat unpredictable! This can be because there is some contention between the threads when they all execute your TestCase simultaneously - for example, say the TestCase makes use of some "shared" resource (e.g. property, datasource), then there can be concurrent updates by the threads, although in your case, I think the delay makes this type of contention less likely, just a guess.
Have you tried running your TestCase with 1 thread, do you get the expected number of runs?
Regards,
Rupert
I set the Runs per Thread to 1 and executed with 5 total threads.
When teststep1 hit 5, the testcase stopped.
If I want to test 3 different users hitting the same REST API, should these be separate test cases within a test suite, each with their own load test associated with them?
Or should it be as I have it now, which is: three separate teststeps, in 1 test case with 1 loadtest testing all three 'at the same time'?
What is the better practice here?
Hi,
I meant to try Threads=1, runs per thread=10 (for example, or bigger), do you get a count of 10 in this case? I am asking as a sanity check, in other words this should result in each TestStep in the TestCase being run 10 times. Then, if this works, increase the number of threads until you notice the counts become inconsistent - this might indicate a thread contention issue.
I would say that, to simulate multiple users (threads) hitting your endpoint, that your load test is correctly setup, so like you say "Or should it be as I have it now, which is: three separate teststeps, in 1 test case with 1 loadtest testing all three 'at the same time'?"
Regards,
Rup
rupert,
Thank you. Upon inspection, I still had quite a lot of debug enabled on the server side hosting the service. Since turning it off, the tests are executing as expected.
I do have another thread/question open about runloadtester.sh and the output it throws to the screen. If you have any experience with the command line execution, I could really use your expertise.
Thank you for all your help & feedback.
~Matt
