MEMORY LEAKS, non suitable for load tests!

Question

Hi All,does anyone use this tool for load/performance testing in&nbsp;REALITY?I really&nbsp;will be glad&nbsp;to hear&nbsp;not that marketing bull**bleep** but real success story...&nbsp;We have implemented quite a lot of functional tests around the SoapUI, even free version covers most of our needs.But it seems that now we came to the line to throw it away (or at least use some other product for load tests).The idea was quite simple - use some of the functional tests as building&nbsp;bricks for&nbsp;load&nbsp;tests.Yes, I know that this is not always an optimal solution (quite expensive to carry on all the asserts as in functional tests) but still, this should be possible as we have enough of hardware to emulate required number of VU, I thought...We decided to evaluate commercial version of ReadyAPI/LoadUI&nbsp;(I would say it costs quite a lot but we CAN offer it if it really works and solves our needs).&nbsp;But evaluation results really disappointed us.Please find attached ReadyAPI project.Observations:- On localhost I have Tomcat running and serving requests to static .xml file (file size is approx 1 kB).- with 10 VUs and zero wait time LoadUI is capable to perform approx 1000 requests per second to this file (while JMeter on the same hardware makes 1800+), ok,&nbsp;acceptable.- in reality&nbsp;we are going to use our existing functional testcases so we test this scenario using one Run Testcase step.this drops TPS to 800.- then&nbsp;add "enormous" groovy Setup script consisting of one "//" (comment). TPS drops to &lt; 200 !- less than a minute of such run and LoadUI occupies all available heap and dies.So&nbsp;even&nbsp;this simple scenario is failed!What to say about complex test scenarios.Is SmartBear able to understand that&nbsp;we can't use buggy tools for QA.Why not to invest some time/money in bugfixing rather than adding new features?Who&nbsp;needs Cloud Testing if memory is leaking during test!Regards,multiware

mfagerlind · Answer

Well, I do know for a fact that it has been successfully used for many&nbsp;pretty massive load tests, so I don't think that's a general problem. We recently proved to a very happy customer that we could generate 100,000 virtual users. And since I'm not in marketing but a developer you may be more inclined to trust me than the guys with pointy hair and/or fancy suits. :-)
&nbsp;
However, you need to use LoadUI agents to build&nbsp;this kind of load (and for that you need the Pro license).&nbsp;
&nbsp;
If you open a ticket with Customer care they will be happy to help you. Forum posts&nbsp;usually&nbsp;won't get as much attention from the Customer care guys.
&nbsp;
Kind regards,
Manne, Ready! API Developer

mfagerlind · Answer

I should add that I will create a bug report and attach your test project. Maybe there's a problem with running load tests with huge Groovy setup scripts. I'll tell the LoadUI team to report back in this thread when they've had the time to look at it.
&nbsp;
And if you want to get this fixed, I still think it's a good idea to open a ticket with the Customer care team.
&nbsp;
Kind regards,
Manne

mfagerlind · Answer

multiware, I assume that you've been using 1.6 when testing. Could you confirm that this is the case (I need it for the bug report).

multiware · Answer

Hi&nbsp;MFagerlind,yes, Ready!API v1.6.0also can you please explain what is "Customer care team".I was not able to find access to any public issue tracker to submit the report.MFagerlind wrote:We recently proved to a very happy customer that we could generate 100,000 virtual users.the number of virtual users actually says nothing about actual test performance&nbsp;util there is info about their scenario.What kind of requests and how often...MFagerlind wrote:Maybe there's a problem with running load tests with huge Groovy setup scripts.As I&nbsp;mentioned in the initial post (and this can be seen in the attached project), the "hudge" groovy script included only one line with "//" (comment) and nothing else.Thanks.

mfagerlind · Answer

Hello again,
&nbsp;
Just to be clear: I'm not saying that you don't have a real problem, or that it's your fault. But we do have a fairly large customer base and people generally don't hit a brick wall the way you do. We have to understand what's biting you.
&nbsp;
Thanks for the info about the version, I'll add it to the bug report.
&nbsp;
To get in touch with our Customer care team, I think filling out the form here is the way to go:&nbsp;https://support.smartbear.com/message/?prod=ReadyAPI
&nbsp;
We currently don't have a public issue tracker, unfortunately.
&nbsp;
Kind regards,
Manne
&nbsp;
&nbsp;

Forum Discussion

MEMORY LEAKS, non suitable for load tests!

6 Replies

Related Content

TestComplete 12 - memory leak (?)

Do not see fake memory leak in a DLL

deploy war with ReadyApi results in memory leak

No suitable driver found for jdbc:sqlserver

Memory leaks

Recent Discussions

How to remove (not clear) test case properties from SaveScript

Remove Properties of TestCase from Groovy.

MyFirstPlugin - button Project menu in Ready Api

Integrate ReadyApi with Devops Release pipelines

Is it possible to integrate the Azure TestPlan Test cases and the ReadyAPI tests ?