Forum Discussion

tdrury's avatar
tdrury
Contributor
13 years ago

calling fail() on LoadTestRunner hangs soapui

Using soapUI Pro 4.5.1 on Windows 7.

I have the following groovy script test step within a test case. The intent is to stop the load test if any thread in the load test gets an error from our server during this particular soap request:


def responseHeaderCode = testRunner.testCase.testSteps['Start'].testRequest.response.responseHeaders['#status#']
def code = responseHeaderCode.get(0).split(' ')[1]
if (code != '200') {
 def responseString = testRunner.testCase.testSteps['Start'].testRequest.response.contentAsString
 log.error 'Thread['+context.ThreadIndex+'] Run['+context.RunCount+']: Start failed:'
 log.error '->'+responseString
 if (context.LoadTestContext != null) {
  context.LoadTestContext.loadTestRunner.fail('load test failed at Start - see logs')
 }
}


This script detects if the test case is being run within a load test and, if so, calls fail() on the LoadTestRunner. This causes an error dialog to pop-up in soapUI telling me the test failed (this is fine), but after clicking 'OK', soapUI is hung. I have no choice but to kill it via the Task Manager.

Initially I tried calling "testRunner.fail('request failed')" but this seems to only fail the test case - the load test keeps running (less the thread that just failed).

Am I doing something wrong? How do I stop the entire load test if one of the threads detects a failure?

-tim

9 Replies

  • tdrury's avatar
    tdrury
    Contributor
    13 years ago
    Sometimes there are reasons for doing non-standard things. It would have been more helpful to try to address my question.

    I connected a debugger to soapUI and walked through the code. For others' benefit (and possibly my future self), here is the answer.

    First, the test case that was being run as a load test effectively was doing this:

    1. start some work
    2. complete some work
    3. if work is not done, go to 1

    What a call to fail() does is set all the other threads to CANCELLED status. This means when the thread finishes running the test case, it will stop. But in my poorly formed test above, the thread could never finish the test case because the "work" would never be done because the failed thread was terminated.

    While I'll admit the test is poorly written, I think soapUI ought to have a better mechanism to kill a load test. It's just not acceptable to hang the application.
  • SmartBear_Suppo's avatar
    SmartBear_Suppo
    Icon for Alumni rankAlumni
    13 years ago
    Hi,

    I am not able to reproduce the issue you are having. When I run a groovy script in a load test with 
    if (context.LoadTestContext != null) {
          context.LoadTestContext.loadTestRunner.fail('load test failed at Start - see logs')
       }


    The load tests stops, shows a pop up dialog, and SoapUI does not hang. Can you upgrade to SoapUI Pro 4.5.2 or 4.6.1? If you are able to attach a project in which this issue occurs that I can try that would help to.


    Regards,
    Marcus
    SmartBear Support
  • tdrury's avatar
    tdrury
    Contributor
    13 years ago
    Marcus, make sure your test case loops - have the last step jump back to the start. Then it will hang.
  • SmartBear_Suppo's avatar
    SmartBear_Suppo
    Icon for Alumni rankAlumni
    13 years ago
    Hi,

    I tried that and I still can't reproduce the problem. Please share what settings you are using for your load test i.e. number of threads, strategy, etc.



    Regards,
    Marcus
    SmartBear Support
  • tdrury's avatar
    tdrury
    Contributor
    13 years ago
    It's a 10 thread, simple strategy which is configured for 1 run per thread.

    Maybe I should try to explain our test (and our app) a bit more. Our app controls factory floors. Imagine you have a 10 step process to build a widget: Step 1 -> Step 2 -> Step 3 -> ... -> Step 10 and each steps adds a few more components on the widget until you have a completed widget after Step 10.

    The guy at Step 2 can't do anything until the guy at Step 1 hands him a partially completed widget. Our test is written so that each thread is acting as a user at a step (10 steps=10 users, therefore 10 threads). The test case we run first checks to see if there is a widget available to process and, if not, sleeps 100ms then checks again. If there is a widget ready to process, it adds a few bits to it, then checks to see if all widgets (we tell it to build 5 widgets) have finished that Step. If not, it goes back to the beginning of the test. If it is finished, the test case ends and that thread is finished.

    If, for example, something really bad happens at Step 2, I wanted to kill the whole load test. But calling fail() within the test case finished thread 2, but thread 3 (at step 3) is waiting for widgets from Step 2 and will never get them. So that thread is never cancelled and soapUI hangs.
  • SmartBear_Suppo's avatar
    SmartBear_Suppo
    Icon for Alumni rankAlumni
    13 years ago
    Hi,

    I am still not able to reproduce this problem. Is there a project you can share that will illustrate this problem?



    Regards,
    Marcus
    SmartBear Software
  • tdrury's avatar
    tdrury
    Contributor
    13 years ago
    Marcus, you would't be able to run the project file as it relies heavily on web services on our app which you won't be able to access.

    I'll work on removing all the dependencies on our app and see if I can reproduce the result with a project I can send you. However, this will take a little time and with the upcoming U.S. holiday, it will probably be a couple weeks.
  • SmartBear_Suppo's avatar
    SmartBear_Suppo
    Icon for Alumni rankAlumni
    13 years ago
    Hi!

    Thank's for the update.

    Let us know when you have more information on this.

    --
    Regards

    Erik
    SmartBear Sweden