Forum Discussion

Occasional Contributor

11 years ago

Does TestComplete Truly run Headless (TestExecute as well)?

So we have one (1) TestComplete License, we have the TestExecute which comes with it.

There are Three machines involved in this scenario, but Only 1 Requires TestComplete

(named myDesktop to develop tests), and 1 (named myVM, a virtual machine) requires

TestExecute to run tests. The third Machine (named myAppMachine) houses the web

software I am testing.

Here's what works so far:

1) I can develop GUI tests and scripts in Testcomplete from myDesktop

*I can run them from myDesktop against an app on myAppMachine (Yay it works)

2) I can install TestExecute on myVM, and drop my tests/projectsuite there and

invoke TestExecute against them. Remember these tests are testing what is on

myAppMachine.

*I can log onto this VM, and successfully run my tests with TestExecute (Yay again)

3) I need to NOT Log into my VM to make this happen. I need to invoke a .bat file on

   the VM from another system (or ps exec or whatever) and launch these tests

   "headless" meaning NO HUMAN HAND clicks go, or types in "EXECUTE", or SEES

    ANY GUI at any time. And yes of course some of these tests are GUI tests (Most in

   fact). That's the point of trying this tool.

   (Screech, Halt, Stop) <- I want this to say "Yay". Please tell me if this is possible.

   Please do tell me if it's impossible so I stop reading about it and spinning my wheels

   for no good reason. That is:

If you know I cannot launch a set of TestComplete Tests at the Completion of a build,

AUTOMATICALLY, on a remote machine, please tell me.

If you know I can indeed do this please tell me that instead and I will continue to try

and build a Test.proj file to launch a remote command to the VM to execute a .bat file

with a TestExecute invokation in there to run the tests.

4) I have read all your stuff at "Remote Tests via Remote Desktop", and "Running

Tests in Minimized Remote Desktop Windows", etc. I have made the Registry entry

for "RemoteDeskto_Suppressed_When_Mimized", etc. and cannot seem to find a

"Headless" answer. Here is what I mean by Headless":

    After my tests (yes GUI tests) are written, dry run, put in place, and connected to a

   script for launching I do not have to touch it to start it, or ever see the GUI. I just

    want to see the results in my email (which I already "Packed" and emailed

    successfully in a little script). My Build completion (Test.proj) will ultimately launch

   them, but right now I cannot get them to run "Headless". Logging in and starting the

tests is not an option for me here so please be forthcoming because I LIKE YOUR

TOOL and I still want to believe it can do this.

If we can get this to work, I can develop a suite of tests. If it works (and continues

to work) we will renew our license next summer (and most likely upgrade it), otherwise

this tool will not serve us. For now I am the sole "Beta" person involved with this. I

write C#/Selenium tests and they run just fine headless (against the same app I am

trying to test headless with your tool). You can use a phantomjs driver (htmlunit driver if you are using Java) and these GUI tests run headless just fine.

BUT I WANT YOURS TO WORK FOR US. You know why? Two Reasons:

1) I am Stubborn and don't give up & more importantly:

2) If we get other "Testers" to use this next year, if we upgrade our license, your I

    interface would be nice for them (not being developers). I can do the scripts and

    other dev stuff for them). MY C# and Selenium works, but other testers are not

    developers so I would have to be the sole developer of ALL the automated tests

    for the rest of my life :(

Thanks, Chris

Colin_McCrae
11 years ago
Running a very similar setup to you - I had tests run automatically, and headlessly, on a VM as part of a build process.

The entire thing was kicked off by TeamCity (the build engine in use for the project). I simply gave them a BAT file to start TestExecute and it did the rest. Project was stored in version control. Application was web based. So run in a browser on a VM with part of the site hosted on a test VM elsewhere.

Entire run happened headlessly (as you mention, this is down to the setup of the VM). TeamCity would pull down the most recent version of the project from Source Control (GIT), dump it onto the VM with TestExecute installed and then start TestExecute using the BAT file I supplied. High level results were also captured by TeamCity and included in the results of the build. Low level results (which were linked to in the TeamCity output) were stored elsewhere on a network drive.

If I wanted to look in on the run (not often as it was overnight) I used VNC to view the remote VM. Using Remote Desktop to connect would cause the run to fail when you closed the remote connection.

So yes, it can be done. I'm not sure about how TeamCity was set up exactly as our build engineer did that part. I just supplied him with all the parts to make it work.

Colin_McCrae
Community Hero
11 years ago
PhantomJS is a totally different animal.

It emulates browser behaviour without actually rendering a physical display. It'll done via JavaScript emulation. Never used it myself.

I've also used cloud computing and Xvfb as a virtual display in Linux with selenium. It also works well (even sends screen captures back to the controller PC - very handy!). But again, totally different animal.

Neither of the above are really relevant to the TestComplete/TestExecute/VM headless running. I'm still convinced it's something to do with the VM setup if it works fine when you run and watch it but fails when run remotely.

Still waiting to hear back from our build guy ....
AlexKaras
Champion Level 3
11 years ago
Summarizing :), based on the info from Colin, the following should work:

-- If the test VM is not started, then TeamCity should start it. Depending on what VM is used, different approach should be used, but all VMs provide an API that makes it possible to manage them (again, see the referenced earlier comment from Hugo);

-- If VM does not automatically log in the test user, mentioned RDP+ utility can do this from the command line (executed by TeamCity);

-- TeamCity agent on the test machine should be started as a regular application (but not as a system service) when the test user logs in. This will provide the agent with the access to the user's desktop;

-- TeamCity agent should start TestExecute and tests. TestExecute will have access to the user desktop because it was started as an agent's spawned process.
claventure
Occasional Contributor
10 years ago
Okay, so we are in business :) My test ran all the way through and passed (not that simple little google one either, rather a very lengthy complex GUI test). I used the -i 2 option to psexec AFTER leaving a user session logged in on the remote system. So all I have left to do is create that actual interactive login at the end of a build (my job, not yours to answer). I am certain I will be able to do that part just fine. For tests that aren't GUI related, TestComplete indeed runs completely "headless" and remote (hands-off). For those that require GUI interaction, having the remote user login established is really an acceptable work around. Thanks and sorry for the trouble, but it was worth a lot to me.
Colin_McCrae
Community Hero
11 years ago
Running a very similar setup to you - I had tests run automatically, and headlessly, on a VM as part of a build process.

The entire thing was kicked off by TeamCity (the build engine in use for the project). I simply gave them a BAT file to start TestExecute and it did the rest. Project was stored in version control. Application was web based. So run in a browser on a VM with part of the site hosted on a test VM elsewhere.

Entire run happened headlessly (as you mention, this is down to the setup of the VM). TeamCity would pull down the most recent version of the project from Source Control (GIT), dump it onto the VM with TestExecute installed and then start TestExecute using the BAT file I supplied. High level results were also captured by TeamCity and included in the results of the build. Low level results (which were linked to in the TeamCity output) were stored elsewhere on a network drive.

If I wanted to look in on the run (not often as it was overnight) I used VNC to view the remote VM. Using Remote Desktop to connect would cause the run to fail when you closed the remote connection.

So yes, it can be done. I'm not sure about how TeamCity was set up exactly as our build engineer did that part. I just supplied him with all the parts to make it work.
AlexKaras
Champion Level 3
11 years ago
Hi Chris,

The short answer: Yes, TestComplete and TestExecute can be used to execute tests unattended (headless).

If your tests do not need GUI, than, most probably, you will have no problems at all.

If your tests need to interact with GUI of the tested program, then you must make it possible for the tested program to output its GUI and at the same time make it possible for TestComplete to access this GUI.

Note, that the above requirement is not related to TestComplete, but is a generic requirement for GUI applications under Windows OS.

My usual answer for the question like yours (you'll find a lot of similar questions on the forum) was that the account that is used for tests execution must have an interactive session on the test box. This session provides the desktop for the tested program, so it (the program) has where to output its GUI, and, at the same time, TestComplete started within the same session, can access this GUI and interact with it.

However, not long ago there was a message from Hugo Lalumiere (http://smartbear.com/forums/f81/t95645/ets,-vmware-and-testexecute/#97066) that could make some corrections to the above explanation in the case of virtual machines.

Anyway, if you need/decide to perform automated logins into the machine where tests are going to be executed, I would recommend to consider the Remote Desktop Plus utility (http://www.donkz.nl/) that I found to work well and to be pretty handy.
AlexKaras
Champion Level 3
11 years ago
Hi Chris,

> "I also have a suite in Visual Studio with C#

using Selenium with the same test steps against the same app I am testing with TC.

I switch to the headless driver Phantomjs and the whole thing is launched remotely,

no GUI "popping" up, yet all GUI objects are instantiated and the test runs it's course

and passes."

I am not a Selenium guru and cannot say for sure, but your note about "headless driver Phantomjs" made me guess about this possible explanation:

-- This driver makes it possible for the browser to start and execute web requests unattended;

-- Also this driver makes it possible for the browser to process all scripts and other staff, so that the expected web page layout is generated;

-- The driver does not execute the actual mouse clicks and keyboard presses for the web page elements, but instead emulates them via the calls to some native DOM methods.

If this my guess is correct, than obviously such tests will be able to be executed without any GUI, just because, in fact, they do not interact with it (with GUI).

TestComplete, on the contrary, is designed to emulate the actions of the human who clicks mouse buttons and presses keys on the keyboard. Thus, TestComplete tests require GUI, interact with it and cannot be executed unattended.

Note: with TestComplete it is possible to create tests that will not directly interact with the GUI but will call DOM methods directly. But this requires that the tests are created in a special way and cannot be swithed on or off with replacement of some driver.

Maybe the above could explain what you observe...

Update: Skimmed the home page for Phantomjs - my guess seems to me to be correct...
Colin_McCrae
Community Hero
11 years ago
Spoke to the build guy.

It's not the actual VM setup, it's how you start TestExecute.

It has to be started by a valid logged in user. It can't be started by a system service. I'm not sure how PsExec does this. But it would explain why it works when you're logged into the VM but not when launched remotely.

So for my setup, TeamCity was seen as the valid, logged in, user so it ran OK.

So check your startup technique I think and see how you get on ....
AlexKaras
Champion Level 3
10 years ago
Chris,

I agree with Colin that something is wrong with your user session and how TestComplete is started.

The reason why first three lines of test succeed is because they don't involve GUI, while the fourth line does involve GUI interaction.

Note, that if you change the third line so that instead of SetText it uses Keys (and Keys() method assumes GUI interaction), it should fail as well.

One more thing: this line

<Exec Command=""C:\Program Files (x86)\PSTools\PsExec.exe" -u domain\username -p password \\192.168.n.n C:\Users\username\Desktop\TestLinkup.bat" />

will execute TestLinkup.bat in the context of the account PsExec service on the target system (\\192.168.n.n) is running at. Usually this is System account and this explains why TestComplete cannot communicate with GUI.

In order TestComplete to start in the user session, the command line must be like this:

<Exec Command=""C:\Program Files (x86)\PSTools\PsExec.exe" -u domain\username -p password -i 2 \\192.168.n.n C:\Users\username\Desktop\TestLinkup.bat" />

where '2' is the identifier of the interactive session on the remote computer where .bat file must be executed in.

There is a problem on how to get the proper session id..., but it usually equals to 2 if only one user is logged via RDP into the target system.
TanyaYatskovska
SmartBear Alumni (Retired)
11 years ago

Hi Chris,

Sounds like you need to run tests via the TestExecute command line. You can specify the needed command-line arguments in PsExec or similar tools. Please refer to the "TestExecute Command Line" article to learn more about it.
Colin_McCrae
Community Hero
11 years ago
I'll see if I can get some info from the build engineer about how the VM was set up. Think that may be your stumbling block. When I had all this set up, he did mention he'd set something similar up at a previous place for automated tests so knew what the VM needed to keep it happy. The VM was part of his build empire so he took care of requesting and configuring it in the ringfenced build domain.

Using VNC instead of Remote Desktop is more for viewing whats going on once the run has actually started.

If he comes back with anything useful, I'll add it here.

** EDIT **

The VM it was run on was running Windows Server 2008 if that makes any difference.