Forum Discussion

tgeliot's avatar
tgeliot
Contributor
8 years ago

TestRunner believes test step has failed when it succeeded

I have a Ready!API project that I've been using for a while.  It gets run at night using TestRunner.

Both my local instance of Ready!API and the one used at night are version 1.8.0.

 

One of the test steps is a Groovy script.  It has worked OK in the past.  The log shows the messages that go with a successful termination of the script.  But now, no matter what I try, Test Runner believes that the test step has failed.  

 

I'll attach my script.

 

But wait, there's more!  Despite the test case having the "Abort on Error" and "Fail TestCase on Error" flags NOT checked, TestRunner declares the Test Case to have failed, and doesn't run any of the remaining test steps.

I'll attach an excerpt of the Jenkins log.  I've had to edit it a bit to remove some sensitive information, replacing test case names with xxx and the like.

 

One possible issue is that there are test steps that run earlier in the test case that DO genuinely fail, and get retried until they either succeed or we give up.  But we want to try all those steps, even if some fail; this is why these checkboxes are unchecked.  Perhaps the failure of an earlier test step is somehow contaminating TestRunner's understanding of what's happening with this test step?

 

3 Replies

Sort By
    • tgeliot's avatar
      tgeliot
      Contributor
      8 years ago

      Yesterday I made two changes:

      1) I created a new Groovy Script Test Step, and copied and pasted the text from the old test step into the new one, and disabled the old test step.  This was just in case somehow the metadata for that original test step had gotten corrupted.

       

      2) I disabled a different Groovy script test step that occurred earlier in the test case, that was always failing.

       

      The next run of the test suite ran OK.

       

      I've installed 1.9 and am running the test now.

      • tgeliot's avatar
        tgeliot
        Contributor
        8 years ago

        What I can say now is that some combination of recreating the Groovy script test step, eliminating the earlier Groovy script test step that really was failing, and going to Ready!API 1.9.0 has at least made the problem go into remission if not fixed it completely.

         

        I'd say that we can declare this "fixed" and I'll re-address the issue if it comes back.