Parallel Tests in SOAP UI PRO 3.6.1
Hello, Parallel run is sometimes broken due to SoapUI re-uses the same testCase object for two parallel tests, making them to use the same values for different test cases. Here's the log: End ...
Forum Discussion
Let me clarify the case:
We use a groovy script that generates unique test data for every parallel test, and then assign them to the testCase object properties:
The generator script is executed from the TestRunListener.beforeRun.
We have noticed that some tests fail randomly.
The investigation shown that some variable values in last step assertions do not match to the same values used in first test steps. Then we suspected that two tests conflict, and started to print testCase instance to log. The results you see above:
- most test cases has their own unique testCase instance
- some though share the same testCase instance, and such test cases fail
This is a major issue for us. We have hundreds of relatively slow tests (up to 30 seconds duration), and the parallel run may decrease the full test suite execution 4 times at least.
We use a groovy script that generates unique test data for every parallel test, and then assign them to the testCase object properties:
testRunner.testCase.setPropertyValue("tn",tn)The generator script is executed from the TestRunListener.beforeRun.
We have noticed that some tests fail randomly.
The investigation shown that some variable values in last step assertions do not match to the same values used in first test steps. Then we suspected that two tests conflict, and started to print testCase instance to log. The results you see above:
- most test cases has their own unique testCase instance
- some though share the same testCase instance, and such test cases fail
This is a major issue for us. We have hundreds of relatively slow tests (up to 30 seconds duration), and the parallel run may decrease the full test suite execution 4 times at least.