Something to watch out for at the beginning of test threads
Something I've noticed recently...
The first web service call of a given thread (for example in a load test) will be a lot slower. If you're recording individual test step times somewhere (for example using Groovy, to correlate inputs with return times) then it's worth knowing.
For example, if I start 8 threads, all hitting the same no-arg web service operation returning a small string, and write the timings (in ms) to a file in the order they return, it will look like this:
This is repeatable with 100 or 200 or n threads, there will always be exactly n very long times coming back, even when the first threads start returning second and third values before the later threads return for the first time. It's not a particular number of milliseconds; larger operations suffer proportionally more but it's hard to quantify so far.
It appears to be a per-thread startup hangover. I hesitate to call it a bug, since getting precise (ie., functional) results out of what is effectively a load test is a bit much to ask, but it's worth making a note of.
J
The first web service call of a given thread (for example in a load test) will be a lot slower. If you're recording individual test step times somewhere (for example using Groovy, to correlate inputs with return times) then it's worth knowing.
For example, if I start 8 threads, all hitting the same no-arg web service operation returning a small string, and write the timings (in ms) to a file in the order they return, it will look like this:
667
729
738
730
643
863
735
737
9
7
7
7
6
8
7
9
7
7
6
8
This is repeatable with 100 or 200 or n threads, there will always be exactly n very long times coming back, even when the first threads start returning second and third values before the later threads return for the first time. It's not a particular number of milliseconds; larger operations suffer proportionally more but it's hard to quantify so far.
It appears to be a per-thread startup hangover. I hesitate to call it a bug, since getting precise (ie., functional) results out of what is effectively a load test is a bit much to ask, but it's worth making a note of.
J
