cancel
Showing results for 
Search instead for 
Did you mean: 

Exception handling good practice in Jscript

Community Leader

Exception handling good practice in Jscript

Hi all,

 

I recently posted a question regarding how to capture the call stack so I can output it to the log when something goes wrong.  What I did discover though is that that's not actually what I should have been asking Smiley Tongue

 

If you are interested:

 

Previous question

 

Now on to the current question.  What is the best way to do robust error handling of try..catch.. using Jscript.  I have an OO background (c#) so naturally tend to go down that route when trying to solve an issue.  That isn't always the right solution for scripting languages though.  From the previous post:

 

function Scriptcall1()
{
  try
  {
    LogErrorStack1();
  }
  catch(e)
  {
   switch(e)
   {
    case "1":
    Log.Message("This isn't serious, just continue");
    break;
    case "2":
    Log.Error("This is very serious");
    break;
   }
  }
}

function Scriptcall2()
{
  try
  {
    LogErrorStack1();
  }
  catch(e)
  {
   switch(e)
   {
    case "1":
    Log.Error("This is very serious this time");
    break;
    case "2":
    Log.Message("This isn't serious this time");
    break;
   }
  }
}

function LogErrorStack1()
{
  try   
  {
    LogErrorStack2();
  }
  catch(e)
  {
    throw e;
  }
}

function LogErrorStack2()
{
  try
  {
    throw("1");
    // or throw("2");
  }
  catch(e)
  {
//***** Here is the actual question I should be asking: ********
    LogException(e,"Perhaps you should try that again"); //But I don't want this to error. I  want to re-throw the exception to the top level function and let that decide if it's a serious problem
    throw e;
  }
}

As you can see, my outlook is to equivalently re-throw an exception up the chain until it reaches a function that knows what to do with it and whether to catch and throw or catch and continue.

Case "1"; Case "2" isn't ideal either, since it leaves an opening for a new "Exception" type to be added down the road that is then not handled.  How do you handle this ?  One of the ideas I'm thinking of is to build a custom exception "class" which is just a KVP with Exception Types which I define in an enum as key and call stack as value.  Can you throw this ?  I'm all ears to hear what you think of my attempts.


-------------------------------------------------
Standard syntax disclaimers apply
Regards,
11 REPLIES 11
Community Hero

Re: Exception handling good practice in Jscript

@tristaanogre Any ideas?

Community Manager

Re: Exception handling good practice in Jscript

Any script guru?

@AlexKaras@shankar_r@tristaanogre, we need your advice.

---------
Tanya Gorbunova
SmartBear Community Manager

Did my reply answer your question? Give Kudos or Accept it as a Solution to help others.↓↓↓↓↓
Community Hero

Re: Exception handling good practice in Jscript

Hi,

 

Personal private opinion based on the projects I participated in:

a) TestComplete provides call stack in the Call Stack log pane. I think that the provided call stack is good enough and is bound to the test code making it possible to navigate to code from call stack. At the same time I don't see any good reason to post a full call stack to the test log itself because test log should be compact, readable and understandable for non-programmers (like manual testers or managers). Thus I don't see the reason in getting call stack from exception.

 

b) The only case when exceptions were useful in test code was for web services testing when the service was called with incorrect parameters in order to get the 4xx return code. In all other cases the called function either reported a problem or returned an indication of success/failure processed by the calling code as required. This was always possible because, by definition, test code always perform defined action to verify defined behaviour and thus always know expected code call result. In case test code fails because of unexpected situation (e.g. when attempting to write to a read-only file), then it is perfectly fine if the code fails. This reveals but not hides the code problem which can be immediately addressed. If it is possible that the given file can be read-only, then test code must be improved to make file writable before writing to it. If the file must not be read-only, then test code must be improved with verification block and clearly report a problem to the test log.

 

Regards,
Alex
[Community Expert Group]
____
[Community Expert Group] members are not employed by SmartBear Software but
are just volunteers who have some experience with the tools by SmartBear Software
and a desire to help others. Postings made by [Community Expert Group] members
may differ from the official policies of SmartBear Software and should be treated
as the own private opinion of their authors and under no circumstances as an
official answer from SmartBear Software.
[Community Expert Group] signature is used with permission by SmartBear Software.
http://smartbear.com/forums/f83/t86934/community-experts/
================================
Community Hero

Re: Exception handling good practice in Jscript


@AlexKaras wrote:

a) TestComplete provides call stack in the Call Stack log pane. I think that the provided call stack is good enough and is bound to the test code making it possible to navigate to code from call stack. At the same time I don't see any good reason to post a full call stack to the test log itself because test log should be compact, readable and understandable for non-programmers (like manual testers or managers). Thus I don't see the reason in getting call stack from exception.

 


Rightly pointed out, As far as automation script error call stack concerns TestComplete call stack on log pane is the best place to see. What  @RUDOLF_BOTHMA  trying is looking to be complicated. I don't think we need to have this complicated logic for automation scripts. In my view, most of the run time error occurs can very well be identified during our test runs which we can cover before running against AUT. Also, most of the error during execution related to the object identification/timing issue which try...catch can't catch. 

End of the day, automation should verify AUT without any false positive and if there is any failure a good framework and standard scripting will help to identify the issue. There will be a nightmare in every script where/how that failed that will be a happy headache to take and resolve by trying various scenarios.


Thanks
Shankar R

LinkedIn | CG-VAK Software | Bitbucket | shankarr.75@gmail.com

“You must expect great things from you, before you can do them”


Extension Available

Community Hero

Re: Exception handling good practice in Jscript


@shankar_r wrote:

Also, most of the error during execution related to the object identification/timing issue which try...catch can't catch. 

Yes, absolutely valid and correct point.

 

Regards,
Alex
[Community Expert Group]
____
[Community Expert Group] members are not employed by SmartBear Software but
are just volunteers who have some experience with the tools by SmartBear Software
and a desire to help others. Postings made by [Community Expert Group] members
may differ from the official policies of SmartBear Software and should be treated
as the own private opinion of their authors and under no circumstances as an
official answer from SmartBear Software.
[Community Expert Group] signature is used with permission by SmartBear Software.
http://smartbear.com/forums/f83/t86934/community-experts/
================================
Community Hero

Re: Exception handling good practice in Jscript

I can't add much more beyond what @AlexKaras and @shankar_r have added.  In my opinion, exception handling for automated testing should have the primary goal of trapping errors in the test.  Sure, if the underlying code is complicated enough that there are code bugs, then you need a bit more.  However, the truth of the matter is that the automated test failed indicating the possibility of a failed test.  At this point, the call stack at the failure point provided by TestComplete is sufficient enough to inform as to where to start the investigation and determine the root cause of the failure.  When you make your automation code TOO complicated, then you spend all your time debugging the automation code and not enough time actually validating/verifying your AUT.


Robert Martin
[Community Expert Group]
Please consider giving a Kudo if I write good stuff
----

Why automate?  I do automated testing because there's only so much a human being can do and remain healthy.  Sleep is a requirement.  So, while people sleep, automation that I create does what I've described above in order to make sure that nothing gets past the final defense of the testing group.
I love good food, good books, good friends, and good fun.

Mysterious Gremlin Master
Extensions available
Community Hero

Re: Exception handling good practice in Jscript


@tristaanogre wrote:

When you make your automation code TOO complicated, then you spend all your time debugging the automation code and not enough time actually validating/verifying your AUT.


And I would like to add even more:

-- Unfortunately, it is pretty rare case when the tested application is designed and documented good enough to make test automation to be done really in parallel with development;

-- In most cases, test automation or correction done in order to match application's changed behaviour is done when the development task is completed;

-- The above means that the more time will be required to put automated test into production, the more time will be spend by manual testers to verify the thing that can and should be verified automatically. And this means increased load on manual testing and its decreased efficiency with manual exploratory verification of complex, corner and non-standard cases.

 

With the above in mind, I think that sometimes it is better to have less perfect (from the classical development point of view) test code in favor of more easily understandable and modifiable one.

The less time it takes to the person who supports test code to put it into production, the more time manual testing has for extended application verification.

 

 

 

Regards,
Alex
[Community Expert Group]
____
[Community Expert Group] members are not employed by SmartBear Software but
are just volunteers who have some experience with the tools by SmartBear Software
and a desire to help others. Postings made by [Community Expert Group] members
may differ from the official policies of SmartBear Software and should be treated
as the own private opinion of their authors and under no circumstances as an
official answer from SmartBear Software.
[Community Expert Group] signature is used with permission by SmartBear Software.
http://smartbear.com/forums/f83/t86934/community-experts/
================================
Community Leader

Re: Exception handling good practice in Jscript

Hi all,

 

Back from leave.  Some good pointers there, thanks.  Responses/updates/queries as follows : Smiley Happy

Just keep in mind the throw("1") and throw("2") in my code is me literally throwing an exception, not putting it in psuodo-code to illustrate the point.

 


a) TestComplete provides call stack in the Call Stack log pane.


Yup, I'm accepting of that point.  The call stack in the log should cover what is required for users.  It's just me as a developer that would find it useful and even that I can handle with debugging if required.

 

Also, most of the error during execution related to the object identification/timing issue which try...catch can't catch.

I think we are all with you on that one.  I think object recognition issues fill up more lines of code in my scripts than the actual testing the script does

 

In case test code fails because of unexpected situation (e.g. when attempting to write to a read-only file), then it is perfectly fine if the code fails. This reveals but not hides the code problem which can be immediately addressed. If it is possible that the given file can be read-only, then test code must be improved to make file writable before writing to it. If the file must not be read-only, then test code must be improved with verification block and clearly report a problem to the test log.

Hmm,   my example is a bit rough and not a perfect representation of what my code does, but it's the best I can think of without trying to show you what my code doesn't do yet because I'm still trying to get input on the best way to do it Smiley Frustrated  Wow, what a messy sentence...

 

What I'm thinking of at the moment is negative confirmation, rather than positive confirmation.  Perhaps if the file can't be written to because if it's readonly the test is a PASS rather than a fail.  (No, I can't think of a case where this would be a test either, but bear with me on this one)  If I write my script to always ensure the file is not read-only, I lose that ability using this script function.

 

Now, if I wrote a script that's very simple and can easily be used by standard users...

 

function WriteLineToTextFile(filename, text)
{
 //get file
//open file
//write line in
//save file
}

Can't get much simpler than that.  A user can call the function from a script or visualiser and it will work.  Unless the file is read-only, in which case it will fail.

 

I could add code to make the file not readonly by finding the file with fileinfo or the like then changing the readonly flag in the same function.  This means I have one function that is entirely dedicated to doing one job and can only be called in one context.  If the code was modified slightly I could use it for more than one context

 

function WriteLineToFileInternal(filename, text)
{
  try
  {
    //get file
    //open file
    //write to file
    //save file
  }
  catch(e)
  {
    throw e;
  }
}

I can now use it in two contexts.

 

function PositiveFeedback(filename,text)
{
  try
  {
    MakeFileWriteable(filename);
    WriteLineToFileInternal(filename,text);
    return true;
  }
  catch(e)
  {
    Log.Error("Unable to write to file");
    return false;
  }
}

function NegativeFeedback(filename)
{
  try
  {
    WriteLineToFileInternal(filename,"Test Text");
    //I managed to write to the file, which I shouldn't have been able to
    Log.Error("You shouldn't be able to do this");
    return false;
  }
  catch(e)
  {
if([e somehow gives away that the file is readonly])
{ Log.Message("The file was readonly"); return true;
}
else
{
throw(e) or return false;
} } }

A user no longer calls WriteLineToFileInternal as before - it's now an internal utility function, but they have the PositiveFeedback function that does exactly the same, but with better error handling.  If any better handling is required, the PositiveFeedback function can be tweaked internally without users needing to know about it.  Does this look like a reasonable implementation?  None of these script individually are particularly complex.  As I said, it's a ramble looking for some feedback coming from an OO rather than scripting paradigm.  Any feedback is welcome.


-------------------------------------------------
Standard syntax disclaimers apply
Regards,
Highlighted
Community Hero

Re: Exception handling good practice in Jscript

Hi,

 

Below are just my comments. Obviously, test code can be implemented with or without using exceptions. This is a matter of code design and what requirements do you have for test code.

 

> The call stack in the log should cover what is required for users.

I would say that users do not care about call stack. What they do care is the report about whether or not the tested function works as required. And this must be clear from test log with the clear and understandable description of what exactly did not work (in case of failure). If call stack is required to figure out what or why did not work, for me this means a poor logging. Thus I consider Call Stack feature of TestComplete to be development means.

 

> I can now use it in two contexts.

-- Nothing prevents you to have three functions: to check if the file is writable; to make file writable and to write to the file. And to call them as your test dictates;

-- Each exception type must be explicitly handled in the calling code, so this does not differ a lot from return code. Generic exception handling is not considered as a good programming practice and is not recommended AFAIK;

-- Exceptions are a good means to either handle unexpected situations (like lost of connectivity) or as a means to report a problem and do not crash the application. But the test, by definition, checks a certain defined behaviour and all deviations must be clearly reported in order to be triaged whether or not they are a real problems. If they are not, then the test code must be enchanced to make these deviations to become possible expected handled behaviour;

-- Exceptions are usually either unrecoverable events (when they are handled at the end of the calling code and do a proper cleanup) or must be handled right after function call to make it possible to retry and/or continue. Considering our example with the read-only file, if the exception is thrown and calling function handles it at the end, the only thing that it can done is to cleanup and report a problem. In almost all cases the current test must be stopped after that (because of unstable environment state) and leave you without any information about whether or not the verified task can be done by the end-user. If the exception (or return code) is handled immediately after the function call, then test code has a chance to make the file writable and retry. As a result, you can get a notification that for some reason the file appeared to be read-only, but the whole functionality remained valid after the file was made writable. I think that this is much more useful result if you need to make a decision about tested application's state but not just report the first problem and stop. (Basically, this is the primary difference between the integration and end-to-end testing and TestComplete really excels with the latter though is perfectly usable for the former.)

 

Regards,
Alex
[Community Expert Group]
____
[Community Expert Group] members are not employed by SmartBear Software but
are just volunteers who have some experience with the tools by SmartBear Software
and a desire to help others. Postings made by [Community Expert Group] members
may differ from the official policies of SmartBear Software and should be treated
as the own private opinion of their authors and under no circumstances as an
official answer from SmartBear Software.
[Community Expert Group] signature is used with permission by SmartBear Software.
http://smartbear.com/forums/f83/t86934/community-experts/
================================
New Here?
Join us and watch the welcome video:
Watch the new Interview
Top Kudoed Authors