Anyway to rerun failed test automatically?

[whuang]

Hello,

I have read some post here and realized that TC doesn't have any build in function to rerun failed test automatically. I found this topic (Rerun Tests) mentioned that QAComplete has the feature to rerun failed tests, but it looks like manual rerun. I was wondering is there anyway, QAComplete can automatically rerun failed tests? Because we usually launch the tests and keep it running overnight, but if the tests finished in the middle of the night with failures, it will just seat there and wait for me to launch the next morning. So I was looking for QAComplete or any other way to automatically rerun the failed tests after the execution is done. Does anyone know anyway to achieve this?

Thanks in advance!

[tristaanogre]

Here's the question I always have on this topic:

Why re-run the failed tests right away? They failed for a reason. Either the automation was created incorrectly and they failed due to coding errors.  Or, they failed because a bug was found and needs to be corrected by the developers.  Either way, re-running a failed test automatically will rarely, if ever, do any real good.  Intervention is needed in both cases in order to get the test run pass.

[whuang]

Thanks @tristaanogre , I don't know why sometimes a test failed at the first round and if I rerun it would pass at the second round without making any change to the code. I have seen a lot of people here had this issue. Maybe it was the performance issue of the virtual machine? what happens the most is object cannot be found, but from the error screenshot I can clearly see that the object is on the page, and I did wait for the page to load. So if a rerun can solve the problem, I hope to get it run by itself ideally.

[tristaanogre]

That's still an indication that something needs to be corrected in the automation.  This is what we "lovingly" refer to here on the community as "timing" issues.  It takes time, sometimes, for an object to appear.  And the automation tries to run as fast as possible unless told to wait or detect or find.  An even if you have those detections and waits in place, they may not be set to wait long enough.  So, while it's an intermittent problem, it's still one that needs to be resolved.  A rerun did not "fix" the problem, just you happened to run it in a situation where the application is not under as heavy a load, where the timing worked out better, etc.  So, you rerun it today and it works... tomorrow it may fail again at the same point for the same reason.  A failed test is a failed test.  Fix the failure THEN re-run.

[whuang]

Yes, I totally agree with you, but if it still reports object cannot be found after I told it to wait for the page to load, I really don't know how to fix the issue then. I try not  to hardcode a time out if it is not absolutely necessary to avoid unnecessary waiting.

[tristaanogre]

Agreed, hardcoding is not the way to do it.  But, obviously, something is going on with the timing.  It might be that the "Wait" you have set is still not long enough.  The "Wait" method for a page has a parameter that you can set to wait for longer than the default time in your TestComplete settings.  It could be that you need to increase that amount to wait longer for the page to load.  Again, keep in mind that's a MAXIMUM wait.  It won't always wait that amount so, as an experiment, you could make it some astronomically huge number (120000 milliseconds which is 2 minutes) and that will at least see if it fixes the problem.  Then you can, at your leisure, tweak it down.

[tristaanogre]

And, honestly, if it takes longer than 2 minutes for a page to load, even if it's intermittant, I'd write that up as a bug.  User experience is key with web apps.  I know that if i'm going to a website, I get really ticked off if it takes FOREVER for something to happen if I'm not given some sort of message like "please wait" or "Processing..." or something.

[whuang]

My codes call the same action 6 times during the test, and they were all using the same default wait for page time, which is 40 sec. Sometimes only the second call complains object cannot be found, sometimes it is the fourth call, sometimes it could be the second and the third call both not finding the object, but the page was loding no problem, I was monitoring the test while it was running, visually I could see the page was loaded in less than 2 seconds, and the TC progress bar at the top right which display "waiting for page to load" went away after 2 or 3 seconds, so I assumed TC detected the page was loaded completly. So in this case how can I force it to wait longer? I gave it 40 seconds, but it only took a couple seconds and then go to the next line of codes.

[AlexKaras]

Hi,

> it still reports object cannot be found after I told it to wait for the page to load

As per documentation, page.Wait() waits for the page to load. I.e. it waits until the last byte for the page request is received from the server. For usual static web pages this is enough.

The bad news is that modern dynamic web pages use a lot of scripting and data obtained from additional resources via additional requests. In the worst case, data load and subsequent rendering is implemented using lazy load technique.

The above means that in addition to wait for the page source to load, you need to wait for all scripts on the page to finish their execution (scripts often drastically change DOM structure of the page) and to wait for all additional data is loaded as well.

Exact algorithm of the wait depends on your tested application.

For example, if you know what element you will work with after page load, you may use this approach:

-- Wait for the page to load;

-- Wait for the element to appear on the page;

-- Optionally, if needed, wait until the element is populated with data;

-- Continue the test.

If you don't know what element you will work with (for example, if you need to delay until complete web page that utilizes lazy loads is loaded), the approach like this should work:

-- Wait for the page to load;

-- Wait for all scripts on the page to finish - this (usually) ensures that all external data is received and rendered;

-- Press Page Down key to scroll page one screen down. This will trigger additional data requests via lazy loads. Repeat the above step to wait until all additional data is loaded;

-- Continue pressing Page Down until some anchor element at the end of the page which is out of page viewport during data load (and thus is not visible on screen) becomes visible on screen;

-- Continue the test.

[whuang]

Thanks @AlexKaras , this helps a lot.