We've been using TestComplete and Jenkins/Hudson for 3 years. Overall, it is a
fairly good match. Jenkins is stable and easy to use. We've created a TestComplete
framework that produces nUnit output for test cases, so they can be processed by
Jenkins. This step is vital if you want to take full advantage of Jenkins.
There are two downsides to Jenkins. One, there are numerous jobs that set up
environments and "glue together" various pieces of testing. When others want
to know the status of testing, it is hard to remove these jobs from consideration
and to aggreate tests run in multiple jobs into one simple view. For this, we created
a results database. Now, automation folks use Jenkins, but everyone else looks
at an external dashboard.
The main downside is that Jenkins is only good when there are simple relations between
test suites/test cases. For example, you have to run installation on a server before
running tests on multiple clients. We are now facing much more complicated workflows
spread across multiple environments, with numerous interactions among all of the test
cases. Doing this with Jenkins will not be practical.
Regards,
G