Missing 'cnt' rows in load test statistics

Hmm, I opened the file in Textpad and I can't see what you're describing. Maybe Excel is messing things up?

Regards

Henrik O

Hi Hendrik,

I'm sure error is there
Please look at the "cnt" column.
If you are counting up the lines, you will see, that the following "cnt" values are missing:
- 14
- 17
- 21
- 34
- 42
- 47
- 50
- 53
- 64
and so on.

Do you know, what I mean?

best regards
nico

Hi Nico,

I see what you're talking about, and it looks odd. Do you by any chance have either an example project that we can use to reproduce this behavior, or more detailed steps to reproduce?

Regards,
Arian
SmartBear Sweden

Hi Arian,

for security reasons, I'm not allowed to send an real project.
But you can also use the soapUI Por sample project.

- create a new load test
- switch from "Seconds" to "Total Runs" and set its value to "1000"
- the other settings are kept as default
- run the load test
- generate a report using "Data Export"


Finaly have a look at the Total*.csv -file

I hope this is enough to reproduce the problem.

Best regards
Nico

Hi Nico,

If you look at your LoadTest Options (click on the tools button), you will see a field called "Sample Interval". The default value is 250 (milliseconds). That's what determines how often SoapUI will write a data-point.

Also notice that the load test defaults have a Test Delay of 1000ms and a Random factor of 0.5. That means that SoapUI will generate a request around every 1000ms + (request time), a little randomized so that the delay will not be always the same. Therefore, the "missing rows" in the reports are actually expected to happen depending on your configuration. To avoid that, try setting a smaller "Sample Interval".

I hope that clarifies matters for you.

Regards,

Renato
SmartBear Software

Hi Renato,

I had some problems to understand your suggestions.
But now I found the time to reproduce and check, what you suggested.
And I'm still not satisfied.

Cnt is the number of Testruns, which is equal to the number of all started threads, isn't it?

So here is my settings:
I set Threads to 5.
I set TestDelay=1000 and Random=0.5 which results in Delays between 500 and 1000 ms.
This means every 0.5 to 1 second 5 new threads are started.

So my questions are:
1) How can be Sample Interval (= 250ms) too large regarding the other settings?
Sampel = 250 < 500 Delay (smalest possible value).
2) I set the Sample Interval to 10 ms (other settings kept unchanged).
I still get empty/missing cnt rows (about 600 rows for 1000 runs, indeed 50 rows more than before, but still not far enough)
How has this value to be set to get the loadtest result for every single testrun (cnt)?

Best regards
Nico

Hi,

2) I set the Sample Interval to 10 ms (other settings kept unchanged). I still get empty/missing cnt rows (about 600 rows for 1000 runs, indeed 50 rows more than before, but still not far enough) How has this value to be set to get the loadtest result for every single testrun (cnt)?

If you change the Sample Interval to 0 you will get all the runs recorded.

1) How can be Sample Interval (= 250ms) too large regarding the other settings? Sampel = 250 < 500 Delay (smalest possible value).

The sample just controls the writing of the statistics to the file. That is independent of the delay in the tests being executed. I\O is more resource and time consuming.


Regards,
Marcus
SmartBear Support

Hi Marcus,

I understand.
Would you please explain to me finaly, why some 'cnt' values are listed multiple times, like shown in the following extract?:

The count value '4' is listed 2 times.

Best regards
Nico

Hi Nico,

I believe the same count appearing more than once is some overlap occurring because of how the statistics are being written to the log. Since specifying 0 means SoapUI Pro will write to the log with no delay the test step may have not been finished, and therefore is appearing twice with it being 4 errors occurring at that time and then 5 errors occurring once it is finished. You could try the Sample Interval using a 0 to 5 range, and see if that is enough to keep the cnt recorded sequentially with no overlap.



Regards,
Marcus
SmartBear Support