Error Handling settings

[kimmatsumoto]

Hello,

I upgraded to TestComplete 14 and noticed a change on error handling during playback.

I setup my default and current setting to handle the On Error as Continue running.

When I looked at my test items (see attachment testeditems) i noticed some times are set to Stop test item and others to Continue running.

Questions:

1. Why was this feature changed?

2. How is it supposed to benefit users?

3. How change I change the Continue running (beside manually)?

Thanks for your help,

Kim

[tristaanogre]

There was an additional option set to Tools -> Current Project Properties -> Playback for "On object recognition".  A new "stop on error" configuration has been added there to allow control whether or not to stop the project on object recognition.  This should answer your question #2: How does this benefit the end user... in that it adds some additional control to "stop on error" configurations.

So, since that logic has changed, I imagine some conversions have been needed in other places. So, there's been some modifications between 12.6 and 14 with regards to the "stop on error" internal hierarchy and priority.  I imagine that in the MDS file that contains those settings, if there wasn't an explicitly set value, it defaults to "Stop test item".   That should answer, as best as I can tell, your first question: Why was this feature changed...new logic requires some conversions.

As for question 3.  Manual, as far as I know, is the only SmartBear sanctioned way of doing it.  However, the MDS file is a structured XML file that you could, if you knew where to find the settings, run a find/replace on the file to change the settings to the desired configuration. 

[tristaanogre]

An additional note:

The settings within project properties handle one level of the "stop on error" logic.  There's a hierarchy that applies in processing "on error" events.  The project level is the overall controller.  Then the test item level.  Setting the settings on the "current project properties" does not apply those settings to the test items.  The "Stop on error" on the test items was there in TC 12.x as well, just that, by default, it was hidden unless you explicityl selected to view the column.

A change in TC 14 (which I find extremely welcome) is exposing that column by default so that the user can actually see what settings are there and make those fine tuned items.

[TanyaYatskovska]

Hi @kimmatsumoto,

Please refer to the following article - it contains a detailed table of the properties' changes implemented in TC 14:

https://support.smartbear.com/testcomplete/docs/general-info/migrating.html#runtime-properties

You will need to update the properties ones.

[DavidMcAuley]

Hello.

So from my understanding is that you are saying that if you have upgraded to testcomplete 14 you will then need to go to each individeal test in test items and manually change the onerror setting from stop test item to your preferred setting you had before testcomplete 14?

If the above is correct and you have say hundreds or thousands of individual test cases then that is a big ask for testcomplete users to do. Was any consideration at smart bear given to fact that testcomplete users may have such situations. I am one such user and yes I can change the MDS files. However the purpose of testcomplete is to make test automation easier. For those who may not have the skill set to change MDS files how is changing On error setting for 100s or 1000s individual test items easy.

David.

[tristaanogre]

That is why "What's new" documents are created, to inform you of changes.  In the What's New for TC 14, we see this: https://support.smartbear.com/testcomplete/docs/general-info/whats-new.html#better-error-handling.  So, to answer "Was any consideration at SmartBear given to fact that testcomplete users may have such situations?" Is... YES.  THAT'S WHY THEY DOCUMENTED IT.  

Just so you know... this actually was something discussed and frequently mentioned by a number of people here on the forums, requesting a better error handlng algorithm to give more granular control over test execution.  So... it was documented... and in the documentation it lets you know "the behavior will change".  It is a welcome change because it allows the developer of the project to be able to more directly control, via test item, when the runner will stop and at what level.

I'm sorry it's inconvenient for you (why am I apologizing?  I'm not a SmartBear employee) but that's why documentation exists so that you are made completely aware of upcoming changes.  Honestly, once you make the changes, it WILL make testing easier.

Happy automating!

[DavidMcAuley]

No one asked you to apologise,.

I get that you are not a Smart Bear employee, thats why you can be rude and condescending. No cost to you.

So if you read the update documenation and dont like its to bad for you and dont upgrade.

Ok I get it. Dont go on the forums so as not be talked down to and treated like an idiot,.

We will give consideration as to whether we should renew our licenses due to your reply.

I will flag it to our local rep.

Cheers.

[TanyaYatskovska]

Hi David,

When opening a project created in previous TestComplete versions, TestComplete 14 converted properties to the most suitable values. However, please pay attention to the fact that the new behavior differs from the previous one. Thus, you may notice different behavior of the test run. In this case, modifying properties is required. Sorry that this bothers you.

The main goal of these changes is to provide you with a better mechanism to handle errors during test execution. I hope you enjoy the new functionality when all migration steps are finished.