Forum Discussion
Not exactly a re-run, but I would write a loop at the beginning of every test case to perform the following steps
1) Check to see if we are connecting
2) Wait to see if the connecting status goes away within a minute
3) If it doesn't go away, kill the application and restart the application
4) If it does go away, continue with the test
5) loop 2-4 in a while loop checking each time through the loop. I'd put a counter in the loop condition as well to limit it that, if it retries more than a certain number of times, mark the test case as failed and move to the next.
Just as a side note... if the application is getting caught in a "connecting" state and it's not an artifact of the test automation, I'd raise a bug to my developers to let them know. Personally, I'd rather not write test automation to work around bugs but let the test automation report bugs. You've found a bug with your automation... it did what it was supposed to do. It's up to the developers to fix it. :) Just my $0.02 US and, in this economy, that might not be worth a lot.
Unfortunately, it is not as straigtforward as it appeared to you...
It is not a developers' issue. They are aware of it. And the issue is not stable in arising. Physical machine does not face it. Win 10 virtual machine faces it rarely... But Win7 and Win8 VMs - that happens a lot.
The issue does not appear in the beginning of a test only. It may appear after any step that results in changing connection status. Many steps may have been performed prior to that.
So simple restarting the app does not work here. I need to start the test from the beginning. And continue the suite.
- tristaanogre8 years agoEsteemed Contributor
There's not a native way within TestComplete to say "rerun a test" because nothing in testcomplete can necessarily be strictly defined as "this is a specific test". Keyword tests can be combined and called from each other, scripts the same. So, to say "re run this" is something that would have to be handled in some sort of code.
I don't know the architecture of how your test project is constructed so I can't say EXACTLY how to do it for you. However, for me, what I would have is some sort of structure wrapped around every ACTUAL test case to detect success/failure and then act accordingly. Additionally, since this is a common problem for your environment, I'd build into every test case a return value that would indicate pass or fail and, if failed, WHY it failed. Then, within the wrapper structure, I'd put in code that, if it detects a failure, check the fail reason. If it failed because of this connection problem, kill the app and loop back to the beginning of the test case. Perhaps this detection process is something that is in a try/catch logic built into every test rather than some sort of calling wrapper, but that's the way I would handle it.
- tristaanogre8 years agoEsteemed Contributor
FYI, you COULD still use my initial suggestion... just a slight modification:
1) Check to see if we are connecting
2) Wait to see if the connecting status goes away within a minute
3) If it doesn't go away, kill the application and make a recursive call to run the test again
4) If it does go away, continue with the test
Now, this is potentially dangerous in that you COULD end up in this massive loop of recursion... you're going to want to detect how many times you're doing this recursion and limit it otherwise, you'll end up with some memory consumption problems.
Ultimately, you want to solve the connecting problem itself. It sounds like it's a problem with the VM environments that are not properly detecting that you've reconnected or something. But, in the meantime, either the framework solution I presented or this one will need to be the way you do things. Personally, I'd prefer the framework solution because it does not present the recursion problem and gives you, ultimately, a structure where you can detect other error types and manage them as well. - Kate8 years agoContributor
Everything you say is correct, good and implemented (this way or another), but does not solve my problem.
Putting every test in a loop does not seem reasonable for me...
From my prospective a test item is a specific test. So what I need is restart a test item. And move to the following item.
- NisHera8 years agoValued Contributor
As far as I understand...........
1) you know there is a problem which may arise in unpredictable manner
2) Once problem arise test hold for a while and stop/finish when timeout
3) System is in undesirable/unknown state, hence you need to star form beginning.
4) You can not go to next test item since "System is in undesirable/unknown state"
well, situation 1,2 and 3 beyond your control. (and probably a bug as tristaanogre mentioned )
But you can control step 4.
you can have system/environment of known state to run next test item.
In automate testing it's a huge risk to test depend of results of previous tests.
if test 1 fails you can not run test 2, if dependent on test 1.
hence my suggestion is keep each tests independent of others. Probably you may need more resources. and may be written so much of code, but do not see any easy way out.
I also believe your developers have to find why connection fails but meantime you can run suite if tests are independent.