QAComplete and test run. Gap in test execution

[azwip]

Howdy folks... ok, have a suite of about 60 tests.... have them scheduled... and executed on remote system....

In the past few weeks... I have noticed an extra long test execution timen for the suite... the suite should run in about 17 hours.... but has been running over 27 hours etc....

Upon investigation, I see, ,randomly, and nor repeatable at will... I see soem tests taking... 2 hours to run, these are 10 to 15 minute tests... but they show show as taking over 2 hours... 

My last run had 59 tests... but 5 of these took over 2-3 hours,,, again, should only take 10 to 15 mins.

I have been unable to determine why yet... it is random tests and times so its not the same tests, and time of day... its random.  

In QAComplete the log displays nothing for those tests when  it happens... all tests before and after, fine and dandy log shows info.... but for the ones that appear to go into the twilight zone... log contains no data.

I look for the test log for these tests in TEST COMPLETE and there is nothing there... those tests dont show.... when that happens to them.

And yes, the next run... next night, those same tests will run fine and it will happen to a handful mof other tests....

Strange.

[Tom_Test]

Hi azwip,

I have a support ticket for something similar.

When you click in QAComplete on the test without a log, is the actual result: "TestComplete (TestExecute) was terminated for an unknown reason."?

I could find the logs on my test client, QAC just didn't get them from there. Same at yours?

Regards,

Tom

[azwip]

Hi Tom, no, in my case... in QA Complete, when I click on a test result for one of the tests that show as it ran over two hours... it simply states. Failed and no records found.

Really weird.

[Tom_Test]

Hi azwip,

our problem startet right after a windows 10 update on our test clients.

[azwip]

Hmmm, I'll have to check into that.... its just really strange... like last nights run, again, a random test... shows it "ran" for almost" 3 hours.... but, in QAComplete, no results... so I can't see what may have happened.... and in TestComplete... under results, that test is not even listed???

But, next run, changing nothing... runs fine and dandy.

[azwip]

So, still not sure what is happening.

Anyone else seeing this?

Every night now... random tests... out of 50, at 3 or 4, show in QAComplete as taking well over 2-3 hours....

but, no info in the log, I have no way to see what is happening.  The test before and the one after, all show fine.

Then, in TestComplete itself, the logs.... do NOT even show those Long running tests (thave have no log at all in QAComplete).

For example, If I have:

Test1

Test2

Test3

   thru

Test 50

Each test normally runs for 10 mins... and their log (pass or fail) can be seen in QAComplete run history and of course in TestComplete

At random, I may see (In QAComplete) as an example:

Test1 - Started and completed (09:00/09:10)

Test2 - Started and Completed (09:10 / 09:21)

   THEN:

Test3 - Started and Completed (09:22 / 13:01) - Over 3hours!!!!

Test4 - Started and Completed (13:02 / 13:11)

etc etc...

QAComplete displays result for all the above... but says no records for that 3+ hour run!  

TestComplete does not show a log at all for Test3 (the 3+ hour run)

In TestComplete I hav logs for all tests, except the ones that run for a ridiculous length of time.... its like they didnt even exist.

Next time i run, those tests (e.g. Test3) will run flawless and the same problem will occur with a different test(s)

I can't even see a log to see what the heck is happening... Lol

Any ideas?

Thanks again!

[Tom_Test]

Hi,

yesterday me and a coworker where able to see this in Testcomplete:

We have an Onstop Event which is just the logoff from our application. The Test ran and we could see the click on the logoff button. Our application went into “no response”. This is where the test are for. Our DEV people are searching for an error in our application. Anyway after several minutes our application window disappears but we could still see the Testexecute widget for about 5 minutes. When the Testexecute widget disappears too as well as Testcomplete. We restarted Testcomplete and where was no log of the Test. Where even were no Windows logs of the crash of our application or Testcomplete/Testexecute. So Testcomplete/Testexecute could as well have crashed first and took our application with it.

Additionally I’ve found false positive tests in QAComplete. The result is shown as passed but if we look into the results, the log stops in the middle of the test and the other test steps are just missing without any error.

So I think Testexecute crashes and QAC don’t get it.

Did you open a support Ticket? It might help Smartbear.

Regards

Tom

[azwip]

Howdy Tom, thanks for the reply and sharing your info as well... in my case I don't "think" its the same situation as per an experiment I tried:

so, I began to wonder if it was QAComplete... or TestComplete... etc...

I created a collection of the exact same tests... in the exact same order.... and ran using ONLY TestComplete this time and NOT QAComplete.... the entire suite/collection, ran, in approx 3.5 hours!!!!  Perfect!!!!

Not one test did NOT run, and not one test took anything near over 10-15  minutes... so, they ran as they should.

So, what I know so far, when running using QAComplete... I have issues and that same suite of tests will take easily over 12 to 14 hours (!!!) ... whan running only using TestComplete, as ran as it should.

I will run that same suite again, this evening, using only TestComplete again and see if it still runs as expected... under 4 hours, no tests missing.

So, so far, "seems" Might be an Issue with QAComplete and my tests...

Have not logged a bug yet... wanted to see if I could gather more info....

[Tom_Test]

Hi,

are you using TestExecute 12.60?