How does the Load Test Limit setting really work?
I've been getting some strange results from my load tests using Thread strategy (other strategies also suffer from the same issue). It seems that if I set the Limit parameter (seconds) to a value that is not long enough, SoapUI will race through the number of threads, and ending up not doing much as shown in the exported statistic history. It seems as if SoapUI doesn't wait for threads to finish before terminating them and launching new threads to fit everything into the allotted time.
The end result is when you export the statistic history, there are only a few rows of stats collected, with many expected rows missing. Specifically, I would expect a number of records for each number of threads ranging from the lower thread limit to upper. However, in my tests, I see only a few rows.
Below's an example stats history export for a test that has Start Threads = 1; End Threads = 60; Limit: 15 seconds (I set this artificially low to make the point):
ThreadCount,min,max,avg,last,cnt,tps,bytes,bps,err
60,15386,15386,15386,15386,1,0,3991,259,0
60,11667,21683,16675,17964,2,0,7982,444,0
As you can see... there are only two records for threads 60. Where are threads 1 to 59? If SoapUI had problems measuring threads 1 to 59, I would expect to see some sort of warning or error messages?
Anyone can shed some light into this? If the Limit needs to be set higher... what is the formula for setting it so that the test works as expected? Obviously, I want to set the Limit to the minimum amount so that the test runs as quickly as possible.
The end result is when you export the statistic history, there are only a few rows of stats collected, with many expected rows missing. Specifically, I would expect a number of records for each number of threads ranging from the lower thread limit to upper. However, in my tests, I see only a few rows.
Below's an example stats history export for a test that has Start Threads = 1; End Threads = 60; Limit: 15 seconds (I set this artificially low to make the point):
ThreadCount,min,max,avg,last,cnt,tps,bytes,bps,err
60,15386,15386,15386,15386,1,0,3991,259,0
60,11667,21683,16675,17964,2,0,7982,444,0
As you can see... there are only two records for threads 60. Where are threads 1 to 59? If SoapUI had problems measuring threads 1 to 59, I would expect to see some sort of warning or error messages?
Anyone can shed some light into this? If the Limit needs to be set higher... what is the formula for setting it so that the test works as expected? Obviously, I want to set the Limit to the minimum amount so that the test runs as quickly as possible.
