Different load test statistics beetween soapUI and Azure web service hosting
Hi, everybody I'm using SoapUI Load testing of my web site deployed on Azure hosting. I've done a load test for 60 seconds and used Thread strategy with params: start threads = 10, End Threads =20 In LoadTest Options I set "Calculate TPS/BPS" as based on actual time passed. That means: TPS: CNT/Seconds passed, i.e. a TestCase that has run for 10 seconds and handled 100 request will get a TPS of 10 As a result of Load test I've got 1.56 tps for my web site. After that I downloaded detail statistic as a CSV file and calculated statistic by hand as CNT/Secons passed and got TPS<0,88 After that I opened SoapUI log file and calculated HTTP responses - 156 for 60sec of test. So TPS is 2,6 After that I opened Azure panel and got 159 HTTP responses for 60sec. So TPS is 2,65 I've executed only one instance of loadtest. Could me anybody answer, why is so different result beetween soapUI test statistics and handmade calculation?
Use groovy to run sequentialy loadtests
I use soapUI 5.2.1. I have TestSuite with 4 TestCases and each TestCase has N test steps. For each TestCase I define one loadTest. My goal it's to run loadtest on each test step to get some stats about the perf of the request. To do that, I use groovy script (run 4 TestCases) : import com.eviware.soapui.impl.wsdl.loadtest.strategy.* import com.eviware.soapui.support.xml.* import com.eviware.soapui.config.* import com.eviware.soapui.model.settings.Settings; import com.eviware.soapui.settings.HttpSettings; import com.eviware.soapui.impl.wsdl.loadtest.*; import java.util.Date; //define constants final int NB_THREADS = 20; final int NB_TIME = 300; def startDate = new Date(); final String PATH_RESULT = "result_"+startDate.format("yyyy-MM-dd-HH-mm")+".txt"; // create groovyUtils and XmlHolder for response of Request 1 request def groovyUtils = new com.eviware.soapui.support.GroovyUtils( context ); def testCases = context.testCase.testSuite.getTestCaseList(); //create burstStrategy def xmlobjBuilder = new XmlObjectConfigurationBuilder(); def xmlobj = xmlobjBuilder.add("burstDuration", "10").add("burstDelay", "0").finish(); fos = new FileOutputStream( PATH_RESULT, false ) String conf = "THREADS : " + NB_THREADS + " TIME : " + NB_TIME + "\n"; fos.write(conf.getBytes()); for (def testCase : testCases){ if (testCase.getLabel().equals("PERF-SIGDOC-RUNNER")){ break; } def loadtest = testCase.getLoadTestByName("perf"); log.info(testCase.getLabel()); fos.write(testCase.getLabel().getBytes()); fos.write("\n".getBytes()); def testSteps = testCase.getTestStepList(); //disable all teststeps for (def teststep : testSteps){ teststep.setDisabled(true); } //call loadtest int i=0; for (def teststep : testSteps){ teststep.setDisabled(false); //set loadtest strategy loadtest.setHistoryLimit(0); loadtest.setLimitType(LoadTestLimitTypesConfig.TIME); loadtest.setTestLimit(NB_TIME); loadtest.setThreadCount(NB_THREADS); Settings settings = loadtest.getSettings(); settings.setBoolean( HttpSettings.INCLUDE_REQUEST_IN_TIME_TAKEN, false ); settings.setBoolean( HttpSettings.INCLUDE_RESPONSE_IN_TIME_TAKEN,false ); def burstloadStrat = new BurstLoadStrategy(xmlobj, loadtest); loadtest.setLoadStrategy(burstloadStrat); //run loadtest WsdlLoadTestRunner loadTestRunner = new WsdlLoadTestRunner(loadtest); loadTestRunner.start(true); loadTestRunner.waitUntilFinished(); //write TPS def statisticsModel = loadtest.getStatisticsModel(); def data = "\t"+statisticsModel.getValueAt(i, 1) +";" +statisticsModel.getValueAt(i, 7) +";" +statisticsModel.getValueAt(i, 10) + "\n" log.info(data); fos.write(data.getBytes()); teststep.setDisabled(true); i++; loadTestRunner.release(); } loadtest.release(); } fos.flush(); fos.close(); With this script , I have a problem . After a moment , the performance fall to zero in terms ofTPS stats. After using the script, I raised the loadloat (without groovy) on TestSTEPS that have inconsistent values , and I get consistent values . Does anyone has any idea about that ?