How to run TestLeft tests on TeamCity

[la2texas]

I've gotten my local environment all working. TestLeft tests execute and pass - wooohooo!

Now I'm trying to build on our TeamCity server, but it fails (Obviously) with this:

Could not resolve this reference. Could not locate the assembly "SmartBear.TestLeft".

What is the process for installing TestLeft on a Build Server Agent and what licensing should one use?

[AlexKaras]

Hi,

It is my understanding, that either TestLeft itself or TestExecute with the enabled REST API module (and several additional assemblies by TestLeft) must be installed on the test machine. (Note, that it is not possible to install both TestLeft and TestExecute on the same machine -- only one out of these two products can be installed on the machine.) Obviously, the installed product must be licensed according either node-locked or floating license type.

I would recommend https://support.smartbear.com/testleft/docs/using/running-tests/from-jenkins.html and https://support.smartbear.com/testleft/docs/using/running-tests/index.html help sections. Despite the fact that the first link is about Jenkins, most of its content should be relevant for TeamCity as well.

[HKosova]

The build agents need:

• TestExecute (the runtime engine for TestLeft tests),
• your compiled test,
• the appropriate test runner (MSTest, NUnit or xUnit.net),
• .NET Framework 4.5+,
• an interactive user session (it can be started by using the SessionCreator utility).

TestExecute is licensed using a floating model, meaning you can install it on any number of build agents, but only N instances can run at a time, where N is your licensed number of seats. You'll need to install the license manager somewhere on your network, and your build agents must have access to the license manager PC.

The TestExecute version must match your TestLeft version, e.g.:
TestLeft 2.1 - TestExecute 12.1
TestLeft 2.0 - TestExecute 12.0

Also note that when you compile your test, you get multiple files in the output:

• your.test.assembly.dll
• SmartBear.TestLeft.dll
• SmartBear.TestLeft.WebApiWrapper.dll
• RestSharp.dll
• ...

You need to copy ALL of these files to your build agent.

Also check the links in Alex's answer for info on how to start the REST server, use SessionCreator, etc.

[AlexKaras]

@HKosova:

Hi Helen 🙂

Thank you a lot for the detailed answer! Though I have one question:

-- Taking that standalone TestExecute (i.e. not the one included in the TestLeft installer) can be used as a runime engine and that

-- The assembly with TestLeft tests can be provided to the third-party customer (who owns TestExecute license)

-- Are there any license restrictions regarding what assemblies from TestLeft's installation (like mentioned SmartBear.TestLeft.dll and SmartBear.TestLeft.WebApiWrapper.dll) can be provided to third-party client ? Can this be found in TestLeft documentation ?

[HKosova]

@AlexKaras,

Good question. I'm not sure of the answer. Could you please ask Support about this? Let us know what you find out.

[AlexKaras]

@HKosova, @la2texas :

Update:

In the course of Support Case #00199864 I was told that libraries and modules specific to TestLeft cannot be shipped to third-party customers (my note: even if they have TestExecute licenses).

I think that it may be a good idea to double-check this with SmartBear Sales if someone was planning to provide TestLeft tests to the customers.

[HKosova]

Thanks for the update, Alex.