How to run TestLeft tests on TeamCity

Question

I've gotten my local environment all working. TestLeft tests execute and pass - wooohooo!

Now I'm trying to build on our TeamCity server, but it fails (Obviously) with this:

Could not resolve this reference. Could not locate the assembly "SmartBear.TestLeft".

What is the process for installing TestLeft on a Build Server Agent and what licensing should one use?

hkosova · Answer

The build agents need:

TestExecute (the runtime engine for TestLeft tests),
your compiled test,
the appropriate test runner (MSTest, NUnit or xUnit.net),
.NET Framework 4.5+,
an interactive user session (it can be started by using the SessionCreator utility).

TestExecute is licensed using a floating model, meaning you can install it on any number of build agents, but only N instances can run at a time, where N is your licensed number of seats. You'll need to install the license manager somewhere on your network, and your build agents must have access to the license manager PC.
&nbsp;
The TestExecute version must match your TestLeft version, e.g.:TestLeft 2.1 - TestExecute 12.1TestLeft 2.0 - TestExecute 12.0Also note that when you compile your test, you get multiple files in the output:

your.test.assembly.dll
SmartBear.TestLeft.dll
SmartBear.TestLeft.WebApiWrapper.dll
RestSharp.dll
...

You need to copy ALL of these files to your build agent.
&nbsp;
Also check the links in Alex's answer for info on how to start the REST server, use SessionCreator, etc.

alexkaras · Answer

HKosova, la2texas&nbsp;:
Update:
In the course of Support Case #00199864 I was told that libraries and modules specific to TestLeft cannot be shipped to third-party customers (my note: even if they have TestExecute licenses).
I think that it may be a good idea to double-check this with SmartBear Sales if someone was planning to provide TestLeft tests to the customers.

alexkaras · Answer

Hi,
&nbsp;
It is my understanding, that either TestLeft itself or TestExecute with the enabled REST API module (and several additional assemblies by TestLeft) must be installed on the test machine. (Note, that it is not possible to install both TestLeft and TestExecute on the same machine -- only one out of these two products can be installed on the machine.) Obviously, the installed product must be licensed according either node-locked or floating license type.
I would recommend https://support.smartbear.com/testleft/docs/using/running-tests/from-jenkins.html and https://support.smartbear.com/testleft/docs/using/running-tests/index.html help sections. Despite the fact that the first link is about Jenkins, most of its content should be relevant for TeamCity as well.

alexkaras · Answer

HKosova:
&nbsp;
Hi Helen :)
&nbsp;
Thank you a lot for the detailed answer! Though I have one question:
-- Taking that standalone TestExecute (i.e. not the one included in the TestLeft installer)&nbsp;can be used as a runime engine and that
-- The assembly with TestLeft tests can be provided to the third-party customer (who owns TestExecute license)
&nbsp;
-- Are there any license restrictions regarding what assemblies from TestLeft's installation (like mentioned SmartBear.TestLeft.dll and SmartBear.TestLeft.WebApiWrapper.dll) can be provided to third-party client ? Can this be found in TestLeft documentation ?
&nbsp;

hkosova · Answer

AlexKaras,
&nbsp;
Good question. I'm not sure of the answer. Could you please ask Support about this? Let us know what you find out.

Forum Discussion

How to run TestLeft tests on TeamCity

6 Replies

Recent Discussions

TestLeft is no longer supported?!?

UI Spy does not open

What is meant by the phrase adapter/connector?

Related Content

TestComplete test results in TeamCity

Start TestExecute from TeamCity

TestComplete and TeamCity Integration