Scott_Tindal
15 years agoOccasional Contributor
Re: Avoiding high memory use when downloading large documents
I'm using soapUI Pro 2.5.1, executing command line loadtestrunner.bat to run a load test that will repeatedly download large (100MB) documents using a web
service call.
- I've modified loadtestrunner.bat to have -Xms1024m -Xmx1024m so that the java heap is increased to 1GB
- The web service supports MTOM, and I have configured the request properties to have Enable MTOM and Force MTOM to true.
- In the testcase options, I have also set 'discards successful testresults to preserve memory'.
It appears that soapUI retrieves the entire document into memory during a download, with the above setting previous downloads can be garbage collected so I don't get OutOfMemory errors, but when running a number of threads, soapUI still uses up java heap memory quite quickly resulting in a lot of time spent in garbage collection (during a 5 thread test downloading 10MB documents, JVM throughput was about 66% (i.e. 34% of time spent in GC)). Is there a way to have soapUI stream this type of download to a temp file rather than storing it all in memory, so that this type of test could be executed with less GC overhead? This has a big effect on the download time.
I tried setting the "Dump File" TestRequest property to a file, but this didn't affect the memory usage, and only one file can be specified so I'm not sure how this property would help with multi threaded test.
Thanks,
Scott Tindal
service call.
- I've modified loadtestrunner.bat to have -Xms1024m -Xmx1024m so that the java heap is increased to 1GB
- The web service supports MTOM, and I have configured the request properties to have Enable MTOM and Force MTOM to true.
- In the testcase options, I have also set 'discards successful testresults to preserve memory'.
It appears that soapUI retrieves the entire document into memory during a download, with the above setting previous downloads can be garbage collected so I don't get OutOfMemory errors, but when running a number of threads, soapUI still uses up java heap memory quite quickly resulting in a lot of time spent in garbage collection (during a 5 thread test downloading 10MB documents, JVM throughput was about 66% (i.e. 34% of time spent in GC)). Is there a way to have soapUI stream this type of download to a temp file rather than storing it all in memory, so that this type of test could be executed with less GC overhead? This has a big effect on the download time.
I tried setting the "Dump File" TestRequest property to a file, but this didn't affect the memory usage, and only one file can be specified so I'm not sure how this property would help with multi threaded test.
Thanks,
Scott Tindal