Need to rerun a test in case of certain error
I am testing a desktop app. Sometimes it sticks in 'connecting' state. If it stays in this 'connecting' state for more that 1 min I break the test execution. Is there a way to make TestComplete re...
Forum Discussion
Unfortunately, it is not as straigtforward as it appeared to you...
It is not a developers' issue. They are aware of it. And the issue is not stable in arising. Physical machine does not face it. Win 10 virtual machine faces it rarely... But Win7 and Win8 VMs - that happens a lot.
The issue does not appear in the beginning of a test only. It may appear after any step that results in changing connection status. Many steps may have been performed prior to that.
So simple restarting the app does not work here. I need to start the test from the beginning. And continue the suite.
There's not a native way within TestComplete to say "rerun a test" because nothing in testcomplete can necessarily be strictly defined as "this is a specific test". Keyword tests can be combined and called from each other, scripts the same. So, to say "re run this" is something that would have to be handled in some sort of code.
I don't know the architecture of how your test project is constructed so I can't say EXACTLY how to do it for you. However, for me, what I would have is some sort of structure wrapped around every ACTUAL test case to detect success/failure and then act accordingly. Additionally, since this is a common problem for your environment, I'd build into every test case a return value that would indicate pass or fail and, if failed, WHY it failed. Then, within the wrapper structure, I'd put in code that, if it detects a failure, check the fail reason. If it failed because of this connection problem, kill the app and loop back to the beginning of the test case. Perhaps this detection process is something that is in a try/catch logic built into every test rather than some sort of calling wrapper, but that's the way I would handle it.
FYI, you COULD still use my initial suggestion... just a slight modification:
1) Check to see if we are connecting
2) Wait to see if the connecting status goes away within a minute
3) If it doesn't go away, kill the application and make a recursive call to run the test again
4) If it does go away, continue with the test
Now, this is potentially dangerous in that you COULD end up in this massive loop of recursion... you're going to want to detect how many times you're doing this recursion and limit it otherwise, you'll end up with some memory consumption problems.
Ultimately, you want to solve the connecting problem itself. It sounds like it's a problem with the VM environments that are not properly detecting that you've reconnected or something. But, in the meantime, either the framework solution I presented or this one will need to be the way you do things. Personally, I'd prefer the framework solution because it does not present the recursion problem and gives you, ultimately, a structure where you can detect other error types and manage them as well.Everything you say is correct, good and implemented (this way or another), but does not solve my problem.
Putting every test in a loop does not seem reasonable for me...
From my prospective a test item is a specific test. So what I need is restart a test item. And move to the following item.
As far as I understand...........
1) you know there is a problem which may arise in unpredictable manner
2) Once problem arise test hold for a while and stop/finish when timeout
3) System is in undesirable/unknown state, hence you need to star form beginning.
4) You can not go to next test item since "System is in undesirable/unknown state"
well, situation 1,2 and 3 beyond your control. (and probably a bug as tristaanogre mentioned )
But you can control step 4.
you can have system/environment of known state to run next test item.
In automate testing it's a huge risk to test depend of results of previous tests.
if test 1 fails you can not run test 2, if dependent on test 1.
hence my suggestion is keep each tests independent of others. Probably you may need more resources. and may be written so much of code, but do not see any easy way out.
I also believe your developers have to find why connection fails but meantime you can run suite if tests are independent.
Points 1 and 2 are correct.
3 and 4 - not exactly...
As for 3:
System is in known but undesirable state... I don't want to wait too long, I'd rather restart the test item.
As for 4:
My tests are absolutely independent. This 'connecting' state does not affect the following tests. It just breaks the current item.
I can increase maximum wait time till 5 or 7 minutes and it definitely will work, but I would like too find a more reliable solution.
Since nothing special can be implemented I will probably increase the timeout...
Kate wrote:
Everything you say is correct, good and implemented (this way or another), but does not solve my problem.
Putting every test in a loop does not seem reasonable for me...
From my prospective a test item is a specific test. So what I need is restart a test item. And move to the following item.
As mentioned, there is no native feature for that. Hence the solutions I presented. Feel free to suggest a product enhancement for it at https://community.smartbear.com/t5/TestComplete-Feature-Requests/idb-p/TestXCompleteFeatureRequests but, in the meantime, you'll need to find some sort of code solution.
