Testcomplete 9: Extremely slow test execution

Hi Alex.



I have been experiencing similar problems as Hugo.   FYI I am not on the version you mention above, but when I check for updates it says none are availble - hence I was not aware there was a newer version and have just posted in a diffrent thread asking when there would be an update.  See attached screen recording.  Please note it is an asf file, I had to rename it as it would not allow me to upload an asf extension, so you will need to rename back to view.



As our license and login details are under our CTOs name, I dont have a password.  Does this mean the only way I can download the update is to ask him to download it for me?



Thanks

Just to clarify, updating to revision 1312 fixed the crashes, but it did not fix the 30 seconds to a minute pauses while looking for an object. It is still very slow, now it just doesn't get slower over time and eventually crash.

I upgraded to the 9.0.1312 release, but still did not fix my slow test execution or recoding on our Java app.  Version 9 takes between 4 and 5 minutes to save our Namemapping file everytime there is a change to it, rebuilding it from scratch each time.  You can watch it in an Explorer window, slowly building the file.  I went back ver 8.7 and it saves the file almost instantaneous

.  The other noticeable slowness is trying to find objects in our app.  I don't know if it's a problem with how it identifies Windows Processes in 9.0, or it has something to do with the new namemapping file, but there are noticeable pauses when going from Window to Window.



This issue has been out there for over 2 weeks now.  Has anyone heard an update from Tech Support on what they have found so far?

Hi Everyone!





Here is a status update for this performance thread.

Right now, there are several performance-related things that are resolved, so if you have the performance issue, make sure to apply the updates below.





Here are the details:





1. Build 9.0.1312 resolves one aspect of the performance problem.





In this build, we have fixed certain problems that were indirectly affecting the performance.

This build does not show up when you Check for Updates from TestComplete.

To install this build, just download TestComplete 9.0 from your customer account (or download the trial package) one more time, uninstall the current version, and install the new package.









2. TCP 6000 port conflict was causing TestCompleteService9 to consume extra CPU time. Patch is available.





Some of our clients have some software running in their networks, which is sending a lot of network packets at ports 6000 and 6001.

These ports are used by our TestCompleteService9 service for communication. When the service receives the packets, it consumes a very high volume of CPU time to process them. This slows things down, and may also look like a performance problem with TestComplete 9.0.





We have a patch for this problem, it resolves the conflict. Feel free to request this patch from the Support Team.









3. Working with Galvin  T to investigate one more scenario.





Galvin is having a problem outside of the two known scenarios, so we are working together to investigate this, and come up with a solution.

I will post an update here once we get some positive results.









If someone else would like to help us fix the problem sooner by providing their application and the tests to run, we will be extremely glad to collaborate - just contact us, and refer to this thread. We will figure out the best way to collaborate on this.









Thanks everybody for keeping this thread alive, and updated with more data and the status of the problem on your side - we are working hard to sort things out!

Here are some answers to the new comments on the topic of the thread:





[To Everson Alex and Hugo Lalumiere]

Alex, Hugo, could you send me the data I mentioned in my message posted earlier in this thread (on 15 Jul 2012, 06:29 AM)? This should help us with the investigation.

Thanks!







[To Kim Hartley]

but when I check for updates it says none are availble

That's right, the updated build does not show up when checking for updates, since it is still 9.0 :(

Does this mean the only way I can download the update is to ask him to download it for me?

There are several possibilities here:

- ask the license holder to download the file for you;

- request a trial version from our web site (you are getting the same installation package);

- ask the Support Team to provide you with an expirable link to the package.





I recommend that you update to the new build as soon as possible - this will make your daily work a little bit easier. Let me know how it goes!









[To Jeremy Crouse]

I upgraded to the 9.0.1312 release, but still did not fix my slow test execution or recoding on our Java app.  Version 9 takes between 4 and 5 minutes to save our Namemapping file everytime there is a change to it, rebuilding it from scratch each time.

We did not hear people reporting this specific problem with the Name Mapping file saving. It looks similar to the problem that was fixed in build 9.0.1312, but if you are using this build, this cannot be the same problem. Could you send me your TestComplete project suite?

Here is a bit of code you can use to benchmark:



Sub PerfTest

    Set objProc = Sys.Process("calc")



    lngTestStartTime = Win32API.GetTickCount

    For x = 1 to 10000

        lngStartTime = Win32API.GetTickCount

        Do       

            arrResult = objProc.FindAllChildren(Array("WndCaption", "WndClass"), Array("Calculator", "CalcFrame"), 5, True)

            If arrResult(0).Exists Then Exit Do

        Loop While Win32API.GetTickCount - lngStartTime < 5000          

    Next

    MsgBox (Win32API.GetTickCount - lngTestStartTime)

End Sub



In TC8.2 the above takes 97.890 seconds, in TC9.0.1312 it takes 170.641 seconds.

Thanks, Hugo! I do see the execution time difference between TC 8 and TC 9 with this code.

Passing it to the Dev Team now to get their input.





Thanks again for the input, Hugo! I will report back when I have some results from the Dev Team.

Hello Hugo!





Our Dev Team has analyzed your example. Here is their assessment:





1. We confirm that there is a performance problem in this specific scenario, and we will be working to improve the performance based on your code example. We can't give a time estimation for a resolution yet.





2. In real tests, the specifics that causes the performance difference in the scenario we are looking at, will not be impacting the test execution performance so much. So, there is no need to worry - this specifics itself will not cause any critical performance degradation.









A big Thanks goes to Hugo for revealing this scenario - we will work to implement some improvements!

Any updates on possible release time frame for a fix to the performances issues in TC 9?

Hi Ted and others!





Here is an update:





The history, and the current results.

We have been closely monitoring all performance-related issues reported in TestComplete 9.





What we are seeing so far is that there were a few problems causing this effect. The problems were not related to each other, and manifested themselves in different circumstances. We have worked with several TestComplete users to investigate the situations, locate the root cause of the problems, fix them, and provide patches to the users. Those patches successfully fixed the performance issues in specific usage scenarios. There are still a couple of on-going investigations.





At that, the analysis we have performed with the help of some TestComplete users, indicates so far that, in most scenarios, TestComplete 9 is faster than TestComplete 8 (yeah it may look strange to see such statement in this thread, but this is what we have heard from several clients so far, including those who are seeing performance problems in specific scenarios). Several situations where this was not the case were confirmed to be issues, and were successfully fixed. These changes will be included in the next TestComplete update, which is going to be released very soon (there is no specific release date set yet, but the estimation is the first half of October).





Since there are a lot of TestComplete usage scenarios, it is possible that there are unknown situations where performance degradation can still be seen. We are extremely interested to continue the collaboration to reveal all these scenarios, and implement the needed improvements!





So, if you are still seeing some performance problems:





1. First of all, check whether it is one of the most common situations I described in my earlier post created on "02 Aug 2012, 11:42 AM" (scroll up) - make sure the updated build is used, and the port-related patch is applied (in case the TestCompleteService9.exe process is consuming a lot of CPU time).





2. If the problem still exists, let's work together to sort things out.

Please open a case with us, and provide us with the information I listed in my post created on "15 Jul 2012, 06:29 AM" (earlier in this thread). In your submission, mention this Forum thread, so we can better track the requests related to this discussion.









Thanks everyone for your collaboration and contribution! It really helps us make the product better!