Test Execution never finishes While executing test cases Parallel ?

[livevox084]

Hi,

trying to trying to run test cases in parallel and Test Execution never shows as finished

Here detailed Problem description:

CSV Input Data File-1: GoogleHostList.csv

www.google.com

www.google.com.ua

www.google.co.uk

CSV Input Data File-2: SearchStringsInputData.csv

Test Automation

API Automation

web Service Testing

Objective:

For each ‘Host Name’ in Input data File-1  , I need to perform search operation for all search strings in input data file -2.

Note: My actiual project is different and it takes(per host and per input value) 2 mins for each set of data.

I have 50 hosts and 10 different input values need to test for each host. Then my total test time will go 50*10*2=1000 minutes;

If I can run 50 test cases in parallel, then I can complete total test in 20 mins.

If I do this search operation sequentially it may take more time and I decided to clone test case into multiple test cases(Equals to number of hosts) and then loop each test case for all given in SearchStrings csv  data file-2.

For this I created  SOAPUI project in the following structure:

Project Strutucre

Master test case is a template test case and this will be  used replicated/cloned for number of hosts available in  HostList.csv file.

Project Structure after cloning after completion

Test Case Replicator groovy script:

Reads CSV Input Data File-1: GoogleHostList.csv

Finds the Number of Hosts

Clones Master Test Case into Number of Hosts times

Updates Each Cloned Test Case Properties

Run Them in Parallel.

 For example, if my CSV Input Data File-1 Contains 3 rows (3 Different Hosts Names) then

Test Case Replicator groovy script Clones the master test case into 3 Copies as below:

1. www.google.com
2. www.google.com.ua
3. www.google.co.uk

Then Replicator script updates the Test Case level ‘hostName’ property in all four test cases and then starts them parallel execution mode.

All three cloned test cases are running very fine.

But test SOAPUI projects is not marking as completed even after the test cases are finished.

If we look at execution status at project level, Test Progress bar hangs at 'TestCaseReplicator' Groovy step forever.

Project Level Run Status

If we look at the test progress at Test Suite level, TestCaseReplicator Groovy step progress bar reached to 100% and Marked as completed, but at project level still hangs at TestCaseReplicator Groovy step.

Test Suite Level Run Status

By design SOAPUI disables the log during the parallel execution.

Even I  tried via command line option using testRunner same issue there as well.

Also this problem blocking next clean-up suite for ever.

How to solve this problem?

Anybody can give quick solution for this?

Sample project  and sample input data files are attached to reproduce the issue.

I am using windows 10 Pro 64 bit Machine.

Tried on SOAPUI Open source as well as Ready API Latest Version

[nmrao]

@livevox084,

Thank you for the details, will go thru and can see if I can come up with some concrete inputs.

However, here is quick feedback on running tests in Parallel.
1. Be careful while running tests in parallel. Because:
a. it consumes the resources of local machine where soapui is running. It directly depends on the number of test cases. If a suite contains 50 cases, and the suite is started in parallel will start 50 threads, I guess. Need to see if that machine is capable.

b. It also impacts on the server that soap calls.

I am not really sure if soapui does the thread pooling internally and not really expert in multi thread coding, know only very little.

2. May be you need to consider running limited threads and after completing reassign the threads to do the next assignment. I mean use of thread pool. It may increase some time, but better than never finishing.

[nmrao]

By the way, what version of soapui are your using? Because, the project shows version 6.0.0 where as the latest released version is 5.3.0.
Please clarify.

[livevox084]

Hi ,

I tested on Ready API 1.3.1, Ready API2.0 and SOAPUI 5.3.0.

With  all above versions I see the same problem.

Now I come for threading issue, leave about fifty threads, it is not working for even three threads.

I am sure SOAPUI is capable of running multiple threads becoz it allows to run test cases and test suites in  parallel mode.

My Application under test is capable of handling more than than 50 threads ,absolutely there is no limitation from application side.

Let me know if anything else required.

Thanks,

Rajendra Prasad.

[nmrao]

Ok.

Here is a sample project which I did few days back and it executes test suites in multi-threaded way from within a Groovy script.

May be it would be helpful, have look at it. You may also implement as you need based on that.

Note that it is not using soapui's multi threading.

https://github.com/nmrao/sample-soapui-projects/tree/master/hybridExecution

Here is the original thread
https://community.smartbear.com/t5/SoapUI-NG/How-to-setup-TestSuite-Priority-for-a-Project-with-mult...

[livevox084]

Thankyou Rao,

You are really great.

I will implement my project as per sample you given .

let you know the results.

Thanks you soooo much for your timely help.

-Rajendra

[livevox084]

Hi Rao,

I looked at your sample script.

---

@livevox084 wrote:

Thankyou Rao,

You are really great.

I will implement my project as per sample you given .

let you know the results.

Thanks you soooo much for your timely help.

-Rajendra

---

Even your script has same problem.

Could you please suggest some other alternative

Thanks,

Rajendra.

[nmrao]

Can you elaborate what you mean?

[livevox084]

Hi Rao,

Sorry for delayed replay.

I was busy with some urgent work.

I tried your example suite.

It has solved little bit but not everything.

In your project "runSuites" step runs disabled the sequential and parallel test suites.

And this project is not waiting until these these sequential and parallel test suites complete.

Which i mean it shows Project level status Finished before the Test Suites finish.

Look at my screenshot:

But definitely it is better than my design because it completing the project atleast.

I appreciate your help.

Thanks,

Rajendra Prasad

[nmrao]

I provided a sample project which was readily available with me and gave it to you thinking it might help and you can apply it for your case, not given you as a complete solution. Glad it has helped.

If you can provide the changed project of your, then I can take look see if I help to improve if possible.