Forum Discussion

multiware's avatar
Occasional Contributor
9 years ago

MEMORY LEAKS, non suitable for load tests!

Hi All,

does anyone use this tool for load/performance testing in REALITY?
I really will be glad to hear not that marketing bull**bleep** but real success story... 

We have implemented quite a lot of functional tests around the SoapUI, even free version covers most of our needs.
But it seems that now we came to the line to throw it away (or at least use some other product for load tests).

The idea was quite simple - use some of the functional tests as building bricks for load tests.
Yes, I know that this is not always an optimal solution (quite expensive to carry on all the asserts as in functional tests) but still, this should be possible as we have enough of hardware to emulate required number of VU, I thought...

We decided to evaluate commercial version of ReadyAPI/LoadUI (I would say it costs quite a lot but we CAN offer it if it really works and solves our needs).


But evaluation results really disappointed us.

Please find attached ReadyAPI project.

- On localhost I have Tomcat running and serving requests to static .xml file (file size is approx 1 kB).
- with 10 VUs and zero wait time LoadUI is capable to perform approx 1000 requests per second to this file (while JMeter on the same hardware makes 1800+), ok, acceptable.
- in reality we are going to use our existing functional testcases so we test this scenario using one Run Testcase step.
this drops TPS to 800.
- then add "enormous" groovy Setup script consisting of one "//" (comment). TPS drops to < 200 !
- less than a minute of such run and LoadUI occupies all available heap and dies.

So even this simple scenario is failed!
What to say about complex test scenarios.

Is SmartBear able to understand that we can't use buggy tools for QA.
Why not to invest some time/money in bugfixing rather than adding new features?
Who needs Cloud Testing if memory is leaking during test!


6 Replies

  • Well, I do know for a fact that it has been successfully used for many pretty massive load tests, so I don't think that's a general problem. We recently proved to a very happy customer that we could generate 100,000 virtual users. And since I'm not in marketing but a developer you may be more inclined to trust me than the guys with pointy hair and/or fancy suits. :-)


    However, you need to use LoadUI agents to build this kind of load (and for that you need the Pro license). 


    If you open a ticket with Customer care they will be happy to help you. Forum posts usually won't get as much attention from the Customer care guys.


    Kind regards,

    Manne, Ready! API Developer

    • MFagerlind's avatar

      I should add that I will create a bug report and attach your test project. Maybe there's a problem with running load tests with huge Groovy setup scripts. I'll tell the LoadUI team to report back in this thread when they've had the time to look at it.


      And if you want to get this fixed, I still think it's a good idea to open a ticket with the Customer care team.


      Kind regards,


      • MFagerlind's avatar

        multiware, I assume that you've been using 1.6 when testing. Could you confirm that this is the case (I need it for the bug report).

  • I've attempted to reproduce memory leaks with an attached project without any luck. The problem could be in groovy scripts itself. I could check another script if you have one. Are you having any problem with this exect script? Could you modify it so it'll have any problmes you're mentioning?