Unique body content when load testing with multiple threads
Using open source SOAP UI 5.5.0 on Windows 10 Pro
I have a REST API which allows me to send a file in base64 encoded form and provide a filename. My POST request looks similar to:
<?xml version="1.0" encoding="UTF-8"?>
<FileSendRequest>
<FileBytes>
PE1lc3NhZ2... (BASE64 encoded string - truncated for brevity)
</FileBytes>
<FileName>filename.xml</FileName>
</FileSendRequest>
When I run this manually it works perfectly. In order to load test I created a test suite and added in a groovy script test step that runs before the POST request with the following:
ThreadIndex = context.ThreadIndex
RunCount = context.RunCount
FileName = "FileName_" + ThreadIndex + "_" + RunCount + ".xml"
testRunner.testCase.testSuite.project.setPropertyValue( "FileName", "${FileName}" )
And in the request changed:
<FileName>filename.xml</FileName>
To be:
<FileName>${#Project#FileName}</FileName>
When I run the load test with one thread and 100 requests it works correctly generating 100 files with filenames:
FileName_0_0.xml
FileName_0_1.xml
FileName_0_2.xml and so on...
If I increase the number of threads to 2 then I see filenames:
FileName_0_0.xml
FileName_1_0.xml
FileName_0_1.xml
FileName_1_1.xml so the filenaming appears to be working as I expected.
i.e. I want a unique filename for each instance of the running thread
However my test or the API seems to lose requests or files, once I increase the thread count in the load test to 2 or more.
With 2 threads, 200ms delay and 100 requests in total I see 1-2 files missing. This loss increases as I increase the total number of runs or threads.
With 1000 requests I see a loss of ~9 files
I am doing a basic assertion of checking for a 200 response code and that never fails.
The issue could be my testing infrastructure or the API itself of course, but I'm not sure how to debug any further (I'm a little new to SOAP UI). Can anyone advise if my test is theoretically sound? The API developer is saying that there is a clash of filenames but I don't understand how that could be when using the script as above. Is there a better way to achieve the same thing?
Thanks