Hi everyone,We have a TestComplete (v8.6) project suite with over 700 automated tests. We are using TestExecute to play these tests back. The issue is that when the tests are executed, it takes a very long time for each test to actually start. The indicator opens up saying "Playback", and it will sit there for up to 2 minutes sometimes before the test actually executes. Once test execution starts everything runs fine, but actually starting each test takes way longer than we need. When spread over 700 tests, this can add over 10 hours to our test runs with nothing actually happening.I have followed all of the performance tweaks in the article on this site, and the machines we are running the tests on are very fast machines. Is there any way to speed up the startup time for each test?

Hello Charles,I would suggest if you or your team could optimize your project in such a way that a project will only contain maybe 50 scripts (tests) instead of 700. The reason why starting up the tests takes too much time is that during pre-compilation time (time after clicking the "Run" button), your 700 script's source code will be compiled. Can you also take a look on how many LOCs are there in each script? This would also be a factor during compilation time.Regards -- Paul

Consider also doing some redesign to, instead of having 700 distinct tests in script code or keyword tests, building a framework of sorts to execute the tests. The code base for such a framework will be considerably smaller and, therefore, start up faster while allowing a greater flexibility to add more tests in the future.

Thank you guys both for your responses.Paul, does this compilation happen only once during the test run or is it before every test? I am using COM automation to drive TestExecute. I only open the project suite before the first test then TFS takes over as far as telling it which scripts to run. The delay occurs between every test in the run though, so I'm curious if it's trying to compile the entire project before every test...?Our scripts vary widely in length depending on the test, but just looking at the metrics... we have 770 script units with 47,454 lines of code so we're averaging about 59 lines per script.Is the number of scripts per PROJECT (not project SUITE) the main issue? Because I could technically split the scripts into separate projects within the same project suite, but that will be a huge undertaking so I want to make sure it will make a significant difference before I attempt it.Robert, we are using TFS as our primary "framework" for writing tests. Our QA testers write test cases in TFS, then we script them (using one script unit per case to keep them separate) and associate the script back to the test case. When playing the tests back, I have written a test "harness" that drives TestExecute to execute each test similar to a unit test. We have written a library of functions that are used repeatedly, but we have to use TFS because that is what our department uses and it allows us to keep a history for each test case.Are there any plans to improve performance in larger projects like this? We purchased 5 licenses of TestComplete Enterprise for this very reason, because we were told that there is no limit to the number of script units a project may contain. While this is technically true, a performance limit of 50 scripts per project for testing a huge application like we are is a major cause of concern right now.

Hello again Charles, looks like your execution process is the bottleneck for the test runs. Per my understanding (and please do correct me if my assumption is incorrect), TFS relaunches the project, say project 1 up to 700, in each run. By concept, the steps followed is the following (if right): Launch TestComplete via COM Open Project. Run Script 1 Close Project Repeat steps 2 to 4 until all the queued scripts have been processed.Bottomline is, compilation happens in each run. (And that's only my assumption). If this is not the case then do let us know and we'll try to pin point the bottleneck.BTW, can you check the antivirus settings of your test machine? It might be scanning your scripts during run time.Regards -- Paul

Hi Paul,Thanks again for your continued input here...I've actually designed the test harness so that it does not re-open the project for each test. So for each test run, it opens the project suite before the first test runs, executes all tests within the TFS test plan that I'm running, then closes the project suite upon completion. I am fairly certain it has at least something to do with the size/number of scripts in the project. The slow test startup also occurs when I run an individual test locally on my machine from within TestComplete, although it isn't near as bad. It is also interesting that the problem doesn't seem to occur if I execute a test directly from the script rather than through the project's test items. Each test has an associated test item in the project, so there are also ~700 test items. I need to execute from the test items though because each test has individual timeout values that I need to utilize.We have an entire lab of computers, some with antivirus software and some without, but the problem seems to occur despite this.

Very slow test startup | SmartBear Community

dzrtfox
Contributor
13 years ago
Cool, I will go back and re-read your blog article detailing your framework. I skimmed over it briefly when you first posted it, but didn't really get into the details. I think I can take some of your ideas and combine it with what I've already got in place and hopefully create something that outperforms what I currently have in place.

Thanks again for your help! You're always a good source of info.
Julia_K
SmartBear Alumni (Retired)
13 years ago
Hello Charles,

Delaying before each test item start (even if the project contains such a great number of test items) is not actually expected behavior for TestComplete or TestExecute.

Could you please use this web form to contact our Support Team and send us your project?

We would like to reproduce the behavior you describe. This will help us find a solution for your issue faster.

Thanks in advance.
dzrtfox
Contributor
13 years ago
Great, thank you Julia!! I will do this as soon as I get back to the office on Tuesday.
pjageronimo
Contributor
13 years ago
Can you let the group know what might be causing the slowness during startup?
Julia_K
SmartBear Alumni (Retired)
13 years ago
Charles,

If you have already sent your project to our Support Team, could you please tell me the ticket number specified in the auto-reply message you were to get after submitting the request? This will help me make sure that we have received your request.

If you have not received the auto-reply message, please submit your request to our Support Team once again.

Thanks in advance.
dzrtfox
Contributor
13 years ago
Hi Julia,

Sorry for the slow reply, the holidays have made things crazy around here.

I have not had a chance yet to submit a ticket. I have to verify with my employer that it is ok to send the project since we are testing an application still under development. I will let you know as soon as I do!!

Thank you for following up.

Forum Discussion

Very slow test startup

Related Content

TC 15.53/54 Slow down

soapUI startup crash

XPath slow

Startup Error

Testcomplete is slow to performs basics operations.

Recent Discussions

How to stop TestComplete from failing a test when unrelated processes crash?

Shared user form between projects and event management

How to add external files to a project?

Unintended actions during keyword test run

Test Execution Summary is null