Forum Discussion

jkempton's avatar
jkempton
14 years ago

Poor performance with SoapUI Runner vs Web Page Runner

Hello Smartbears,

I have a simple LoadUI test suite that has a fixed load generator (40 users), and a soapUI runner.

If I run that (the soapui runner does a rest/json get) via SoaUI runner I get ~ 20tps per loadui-client. So if I have 12 clients I get 140 tps. I have tried adding clients and the improvement is always around 20tps per loadui-client...

So I tried making the same call using web page runner and I get much better performances.. now I get ~ 1000 tps and I can lower the number of loadui-clients (too a point) and I see the same performance.

I also need to simulate clients calls that are rest/json via POST, but I can't do that with the webpage runner.

So any ideas how to break though this loadui imposed bottleneck?

Thanks,
John.

10 Replies

  • SmartBear_Suppo's avatar
    SmartBear_Suppo
    Icon for Alumni rankAlumni
    14 years ago
    Hi John!

    Making the same requests in the soapUI Runner and the Web Page Runner will always be a bit more efficient in the latter, since it doesn't have to create a soapUI runtime. That being said, the difference you are experiencing sounds strange.

    Would it be possible for you to send us your projects to support@eviware.com (with a reference to this topic)? If not, please provide as much as you can and compliment with screenshots of your soapUI TestStep as well as loadUI Project.

    Thanks for helping us with this!

    Henrik
    SmartBear Software
  • alibaba82's avatar
    alibaba82
    Super Contributor
    14 years ago
    Hi Henrik,
    If the soapUI testrunner is not very efficient, would it be possible to have a native loadui component for
    - SOAP request
    - REST request (i guess we can just use web page runner for this)
    - JDBC request


    Thanks

    Ali
  • AndyHughes's avatar
    AndyHughes
    Regular Contributor
    14 years ago
    This worries me now. What kind of effect does having a soapui runner in loadui have on the request timings.?
  • SmartBear_Suppo's avatar
    SmartBear_Suppo
    Icon for Alumni rankAlumni
    14 years ago
    Hello everyone!

    Our experience and goal is that the soapUI Runner should be fast enough. LoadUI runs soapUI TestCases in the soapUI engine, so there are no reasons for the performance to not be the same. I have myself made 6000 requests per second in the soapUI Runner (very simple TestCase) without even distributing, so the soapUI Runner's performance shouldn't be an issue. John's performance issues surprises me a lot and unfortunately we haven't heard from him (to have a look at his project, etc).

    Regards

    Henrik
    SmartBear Software
  • SmartBear_Suppo's avatar
    SmartBear_Suppo
    Icon for Alumni rankAlumni
    14 years ago
    A quick update: We have now gotten the project from John and we're not getting any strange limitations in speed/load. We'll continue assisting him in isolating why he's experiencing this.

    /Henrik @ SmartBear
  • NSeibert's avatar
    NSeibert
    Contributor
    14 years ago
    I have also noticed a marked differnce in running tests with the soapUI runner. In particular from an agent.

    My setup

    Windows Cloudserver same instance running both
    LoadUI Agent
    SoapUI

    I run the same soapUI test from both loadUI and soapUI and it is very easy to see that test run soapUI, runs much more of a load than the loadUI agent.

    The loadUI agent drops off and begins to queue up request and then that are discarded all together.

    The same test run with soapUI run pushes a significantly greater number of transactions through the pipe.

    Again, both are running the same test.

    Are there any suggestions on how I mght be able to remedy this situation?
  • SmartBear_Suppo's avatar
    SmartBear_Suppo
    Icon for Alumni rankAlumni
    14 years ago
    NSeibert wrote:
    I have also noticed a marked differnce in running tests with the soapUI runner. In particular from an agent.

    [...]

    Are there any suggestions on how I mght be able to remedy this situation?

    Hi,

    We haven't heard of this before this topic was created, and we haven't been able to reproduce it.

    Care to share any figures, screenshots, videos, projects, etc.? For example, how high TPS do you get in soapUI vs. loadUI local mode (please also provide a screenshot of these two scenarios)?

    Regards

    Henrik
    SmartBear Software
  • SmartBear_Suppo's avatar
    SmartBear_Suppo
    Icon for Alumni rankAlumni
    14 years ago
    NSeibert: Also, are you using HTTPS (it has turned out that the topic creator only experiences problems with HTTPS)?
  • MMeltinis's avatar
    MMeltinis
    13 years ago
    SmartBear Support wrote:


    We haven't heard of this before this topic was created, and we haven't been able to reproduce it.

    SmartBear Software


    Hi,

    We are having the same experience with our project. Running the testcase directly from SoapUI we are able to have 60 r/sec but from within LoadUI we have as few as 20 r/sec when the response time rises to over 60 sec (the test is against another computer running mock services)...it seems that the break point is around 15 r/sec.

    If needed we can send you the project which include both mock services and test cases.
  • SmartBear_Suppo's avatar
    SmartBear_Suppo
    Icon for Alumni rankAlumni
    13 years ago
    MMeltinis wrote:
    If needed we can send you the project which include both mock services and test cases.

    Please do (use this form and refer to this forum topic)!

    Also, which versions of LoadUI and SoapUI is this?

    Regards

    Henrik
    SmartBear Software