Testcomplete 9: Extremely slow test execution

Hi Everyone!





The performance issue is definitely not the thing that was expected from the new release!





So, we need your help to identify specific scenarios when this situation can be seen. If you are seeing unexpected performance decrease in your tests execution, please send us the below information, so we can work on the problem. If you can't provide all details at once, no problem - send us what is available, and we will help you collect other details if you need assistance with this.





Here is the information that we need at this step:





1. Approximate performance difference - execution time in version 8, and execution time in version 9 for the same set of operations.





2. The system information file from the PC where you are running the tests: run msinfo32.exe (Start | Run...), and save the details to an *.NFO file.





3. Tested application details (those, which are available):

  - IDE / programming language used to create the application;

  - main technology used in the tested application (ASP.NET, Flex, .NET WinForms, MFC);

  - predominant third-party UI controls used in the application (e.g. jQuery, Google Web Toolkit, DevExpress, Infragistics, etc.);

  - any application specifics related to how it is running: under a different user account, with restricted permissions, through a runner program as opposed to running the app's executable directly, via ClickOnce or Java Web Start, etc.





4. If available, the description of specific functional areas that demonstrate performance degradation.

For example: several specific Test Items are working much slower, while others are working at the same speed of faster; or all actions over a specific set of controls are slower (all grid controls, or all menu controls).





5. Zip your entire project suite folder, and send it to us.

If it is too large for an e-mail, indicate this in your case description, and we will send you an FTP location where you can upload the files securely.





6. If the test execution results from TestComplete 8 are still available, please send them to us as well - we will compare them against the execution results from version 9.0 to detect specific areas where the performance got lower.









NOTE:

When submitting your case, make sure to do a couple of things:

(*) include a link to this thread - this will help us easier process the case;

(**) set the Severity field to Urgent - we want to resolve this as soon as possible.









I am sure that with your collaboration, we can improve the performance in the problematic scenarios! Thanks!

We have started migrating a very huge project (around 500 parent test items with about 10 children each on average) from TC8.2 to TC9, what I see is actually INCREASED performance in most case, however sometimes TC seems to "hang" doing nothing at all for 30-45 seconds, and then starts over again. It seems to be linked to the .FindAllChildren method, .FindChild method, and some others. Once in a while it won't find the object right away, and will go through whatever depth was selected, go all the way down, and finally return the correct window (which was sitting at the top of the hierarchy).



TC9 also seems very unstable at the moment. It routinely closes on us without warning, sometimes gets "stuck" after stopping a test execution, leaving us with no other choice than killing the process, becomes very slow after some hours of work, and consumes an increasing amount of memory over time. This to me points to a memory leak somewhere. Fresh after a reboot, it is much faster than TC8.2, but this quickly degrades after multiple runs.



We also encounter various errors in our tested application, all linked to TcVbHook.dll in the resulting stack trace, most common being runtime error 91. Our application is about 50/50 VB6 and VB.Net, with some C#.



I am not too worried about it for now, as we only have about 10% of the project converted to TC9, and this is the first release of the 9.0 branch, but I sure hope those issues are resolved by the time we are ready to deploy it to our test pool. We use TC to test and resolve the application's issue, not to create new ones.



Hugo Lalumière

Dynacom Technologies Inc.

Quick update, the crashing to desktop and memory leak seems to come from the object browser. Where I normally have to shutdown everything and reopen every hour or so, I have removed the object browser tab from TC by right clicking the tab and choosing Hide, and have not had a crash in 3 hours now. It still is slow after a while but at least it doesn't die anymore.

Hello Hugo,





There was an issue in the original 9.0 release that revealed itself exactly like this - problems when working with TestComplete for a long time. We fixed the problem, and put the updated build to the web site. Could you please download TC 9.0 one more time - you will get the updated build now.





The build number is 9.0.1313 - it's a new build of the TC 9.0 release, and only this build is now available. Before you install the new build, please uninstall the current version.





Let me know if the problem can still be seen in the updated build.

That fixed it, no more slowdown at all, everything seems to be running smooth for now. I did try to auto-update before, but I guess that first build was a fluke and no update package was made for it.



Thanks for this solution.

Hi Alex,



Is this 9.0.1313 release supposed to be available to everyone?  When I log into my account, I only see the 9.0.1312 release available for download.



Thank you.

Hugo, thanks for the confirmation! That's great to hear the product runs better now!





Galvin, I am sorry - that was a typo in my response.



So, an update:

The correct build number is 9.0.1312, it is available to everyone, and we recommend installing this newer build to avoid stability issues.



Thanks for catching the typo, Galvin!

Actually it was revision 1312, I was updating from 0679 or something like it.

Alex,



Even after upgrading to 9.0.1312, I am still experiencing test execution slowness.  I have submitted information as requested above (Issue M0102218), but still need instructions on where to upload larger files.  I also forgot to mark the issue as "Urgent", so can you change that for me?

Hi,



We have also noticed a decrease in performance after upgrading from TC 8.70 to TC 9.0.1312.7.

The same project that usually took 32 minutes to execute now takes more than 1h20m.



We checked the performance tips, but it didn't help.