Return False from a KeywordTest on any Error

[RUDOLF_BOTHMA]

Hi all,

I'm on a mission to make my tests run endto end without stopping because there was one failure.

I know you can configure TC to continue/stop on error, but what I'm trying to acheive is to be able to run consecutive KWTs and if one KWT fails, all the KWTs that follow that are dependant on this KWT will get cut out, but the ones that aren't will still be run. e.g.

KWT CreateOrder:

Navigate to page -> enter data -> save ->return true/false

KWT VerifyBasket:

Open basket page -> check fields ->return true/false

Now, you have KWT OrderProcess:

KWT CreateOrder

if(lastresult==true)

   KWT VerifyBasket

KWT ChangeProfile

As you can see, KWTChangeProfile is not dependant on KWT CreateOrder at all, so should still run, it's just KWT VerifyBasket that needs to be left out.

Now...

I can write a KWT that returns a true/false at the end of the KWT, but how do you handle if one of the steps halfway down the KWT logs an error ?  If TC is configured to stop, KWT ChangeProfile won't happen.  If TC doesn't stop, KWT VerifyBasket will error all over the place and keep wasting time trying to find objects that don't exist etc.

It's not reasonable to check each and every single possible error in all environments on each step of the KWT and return false, so is there a way to tell my KWT onError, return false ? 

Could it be as simple as wiring up an event like so ?

function GeneralEvents_OnLogError(Sender, LogParams)
{
   return false;
}

[tristaanogre]

Honestly... depends a lot on what kind of error... sure, it logs an error... but is it an object recognition error? Is it a checkpoint that fails that a value is incorrect? There are any number of things that can log an error.... 

...what you need to do is get more DELIBERATE in logging errors and trapping such things if you really want to make it bullet proof.  And that just gets stupid after a while. You need to do SOME of this, but for the "unexpected" stuff, you'll need to trap in an event.

Rather than just simply returning "false", what I would do is have that event handler set a global variable... something like "current test status" to "false" if the test is failing.... then, at the end of the test, in like a finally block, check that value and act accordingly.

[AlexKaras]

Hi,

In addition to what was suggested by Robert, you may consider structuring your tests as a tree of Test Items, set their behavior using the On Error property of Test Item and, in addition, consider the use of Runner.Stop(true).

[RUDOLF_BOTHMA]

---

@tristaanogre wrote:

Honestly... depends a lot on what kind of error... sure, it logs an error... but is it an object recognition error? Is it a checkpoint that fails that a value is incorrect? There are any number of things that can log an error.... 

---

I'm currently only considering errors I generate myself in scripts.  For now I'm happy to let TestComplete generated errors slide.  My scripts go to great lengths to avoid these e.g. by putting in Waits, testing for existence, refreshing if required - and I continually refine them, so they should gradually go away (we hope).  Thing is though if my own script can't find it either I put in

Log.Error("Your KWT called a script with invalid parameters and you are being a muppet");

So, heres a thought.  See what you think.

a. Set TC options to stop on test item

b. Use the AdditionalText in the LogError to communicate where the error was logged from e.g. KWT name if it's under my control

 

c.  Push a KVP in the global variable array for additionaltext->errorState

d. Create a script that, given a text parameter can look into this global variable's KVP array for my text and return the error state

e. my next KWT calls this script with the additionaltext of the prerequisite KWT/script and can then behave accordingly.  If nothing is returned, there is no error at present.  This will also give me the ability to build in multiple prerequisite checks

Limitations/issues I can think of:

If the error comes from a script, it won't know which KWT it was called from so all KWTs will need a try.. catch.. finally and I don't know how to get that additional info from the error logged into the KWT's finally..  I can think of some messy footwork where the error itself (in OnLogError) updates a global variable that says that at this immediate point in time there is an error and this is the error details (from additionalText), then a seperate method called in the finally of the KWT gets that error additional info from the variable, sticks it into my global errorstate array and cleans the current error

---

@tristaanogre wrote:

what you need to do is get more DELIBERATE in logging errors and trapping such things if you really want to make it bullet proof. And that just gets stupid after a while 

---

With you on that one.  That's why I'm trying to get a way (not away ) to elegantly handle the errors I deliberately raised myself at the same time as application errors such as object recognition and be able to bake in prerequisite passed KWTs/scripts into my KWT trees as @AlexKaras puts it

[tristaanogre]

I created a very quick little "throw" routine in JavaScript that I call within my keyword tests (since a "throw" doesn't exist as a default action).  The "Catch" in the KeywordTests then has the option of trapping the data from the "throw" so you can then use it in the catch block.  Perhaps that will help.  

[RUDOLF_BOTHMA]

I like the idea.  Something like this ?

function ThrowException(message)
{
  try
  {
    throw message;
  }
  catch(e)
  {
    throw e;
  }
}

I would prefer if I could get the function's message back as value, rather than TCs Exception occurred and not sure how to get hold of that.