Forum Discussion

claventure's avatar
claventure
Occasional Contributor
11 years ago

Does TestComplete Truly run Headless (TestExecute as well)?


So we have one (1) TestComplete License, we have the TestExecute which comes with it.



There are Three machines involved in this scenario, but Only 1 Requires TestComplete

(named myDesktop to develop tests), and 1 (named myVM, a virtual machine) requires

TestExecute to run tests. The third Machine (named myAppMachine) houses the web

software I am testing.



Here's what works so far:



1) I can develop GUI tests and scripts in Testcomplete from myDesktop

   *I can run them from myDesktop against an app on myAppMachine (Yay it works)



2) I can install TestExecute on myVM, and drop my tests/projectsuite there and

    invoke TestExecute against them.  Remember these tests are testing what is on

    myAppMachine.

  *I can log onto this VM, and successfully run my tests with TestExecute (Yay again)



3) I need to NOT Log into my VM to make this happen.  I need to invoke a .bat file on

    the VM from another system (or ps exec or whatever) and launch these tests

   "headless" meaning NO HUMAN HAND clicks go, or types in "EXECUTE", or SEES

    ANY GUI at any time.  And yes of course  some of these tests are GUI tests (Most in

    fact).  That's the point of trying this tool.

   (Screech, Halt, Stop) <- I want this to say "Yay".  Please tell me if this is possible.

   Please do tell me if it's impossible so I stop reading about it and spinning my wheels

   for no good reason.  That is:



   If you know I cannot launch a set of TestComplete Tests at the Completion of a build,

   AUTOMATICALLY,   on a remote machine, please tell me.



   If you know I can indeed do this please tell me that instead and I will continue to try

   and build a Test.proj file to launch a remote command to the VM to execute a .bat file

   with a TestExecute invokation in there to run the tests.



4) I have read all your stuff at "Remote Tests via Remote Desktop", and "Running

   Tests in Minimized Remote Desktop Windows", etc.  I have made the Registry entry

   for "RemoteDeskto_Suppressed_When_Mimized", etc. and cannot seem to find a

   "Headless" answer. Here is what I mean by Headless":



    After my tests (yes GUI tests) are written, dry run, put in place, and connected to a

    script for launching I do not have to touch it to start it, or ever see the GUI.  I just

    want to see the results in my email (which I already "Packed" and emailed

    successfully in a little script).  My Build completion (Test.proj) will ultimately launch

    them, but right now I cannot get them to run "Headless".  Logging in and starting the

   tests is not an option for me here so please be forthcoming because I LIKE YOUR

   TOOL and I still want to believe it can do this.



If we can get this to work, I can develop a suite of tests.  If it works (and continues

to work) we will renew our license next summer (and most likely upgrade it), otherwise

this tool will not serve us.  For now I am the sole "Beta" person involved with this.  I

write C#/Selenium tests and they run just fine headless (against the same app I am

trying to test headless with your tool).  You can use a phantomjs driver (htmlunit driver if you are using Java) and these GUI tests run headless just fine. 



BUT I WANT YOURS TO WORK FOR US.  You know why?  Two Reasons:

1) I am Stubborn and don't give up & more importantly:

2) If we get other "Testers" to use this next year, if we upgrade our license, your I

    interface would be nice for them (not being developers). I can do the scripts and 

    other dev stuff for them).  MY C# and Selenium works, but other testers are not

    developers so I would have to be the sole developer of  ALL the automated tests

    for the rest of my life :(

Thanks, Chris


  • Running a very similar setup to you - I had tests run automatically, and headlessly, on a VM as part of a build process.



    The entire thing was kicked off by TeamCity (the build engine in use for the project). I simply gave them a BAT file to start TestExecute and it did the rest. Project was stored in version control. Application was web based. So run in a browser on a VM with part of the site hosted on a test VM elsewhere.



    Entire run happened headlessly (as you mention, this is down to the setup of the VM). TeamCity would pull down the most recent version of the project from Source Control (GIT), dump it onto the VM with TestExecute installed and then start TestExecute using the BAT file I supplied. High level results were also captured by TeamCity and included in the results of the build. Low level results (which were linked to in the TeamCity output) were stored elsewhere on a network drive.



    If I wanted to look in on the run (not often as it was overnight) I used VNC to view the remote VM. Using Remote Desktop to connect would cause the run to fail when you closed the remote connection.



    So yes, it can be done. I'm not sure about how TeamCity was set up exactly as our build engineer did that part. I just supplied him with all the parts to make it work.
  • "IT DOES NOT WORK ON GUIS UNATTENDED, REMOTE, HEADLESS"



    But it does? I have it running just like that (running against several websites and multiple browsers). Was minimum hassle to set up as well, but I think that was largely down to the build guy having seen problems just like this so having the VM set up and structured in such a way as to avoid problems.



    Was anyone remoted in, and subsequently came out of the RDP session, when the run was started?



    If you go in via RDP, it will break the user session set up by the automatic start (assuming you have a reliable way of starting it now so it has a valid user session). Using VNC I can drop in and out during a run as I please and it doesn't affect the session ID on the remote VM.



    May be easier to use VNC at least to watch it live safe in the knowledge that it's not your presence in the VM that's causing it to fail. Maybe get a better idea whats failing?
  • claventure's avatar
    claventure
    Occasional Contributor
    Ok wow - frustrated, but that was good to hear "Valid User Session".  So I psexec in "hands-off the system" it kicks off the .bat which launches TestExecute, first line in Keyword test is Browser plus URL to site (works), 2nd line examine title and assert (works), 3rd line place data in a search box (works), 4th line click the search button (Fails), here is the error:

    The operation cannot be performed, because the user session is non-interactive.

     


    Possible cause:




    • The test is run from a scheduled task with the Run whether user is logged on or not option enabled.



    Possible solution:




    • Enable the Run only when the user is logged on option in the scheduled task and ensure the user is logged on when the task is triggered.





    All of this was headless and it ran right up until it had to click a button, implying user must be Logged in, but user was logged in via psexec which is why I got through first 3 lines of keyword test, but maybe that is not valid user login enough to click the search button.  GUI is rendered nowhere.  Still your words are encouraging.  what is most frustrating is that I can do an entire GUI runthrough, headless, remote using Visual Studio c# and Selenium because it allows me to pick a driver (IEDriverServer or PhantomJS for headless).  So I can write all my tests in C#/Selenium just fine headless, but we WANT TO USE TestComplete and make that work as well.  Thanks

     


  • claventure's avatar
    claventure
    Occasional Contributor
    The attachments above show a simply example heading out to the google page, headless, on a VM, with a remote launch of a bat file containing testexecute invocation.  I was thrilled to get this far, but now need to get past the error shown.  I used google instead of our own proprietary stuff as an example.
  • claventure's avatar
    claventure
    Occasional Contributor
    Wow - thanks everyone for helping with this.  I will try this at 10:00 EST.  Your participation is quite valued here and I really need this product to work as the TestComplete interface is very "friendly" to "Non-developer" test types and I really don't want to try and train them up on my Selenium/C# version of testing in Visual Studio, rather I would rather teach them how to use THIS in June.  For now I am the sole "automated" tester doing proof of concept test development and while my own suite (Selenium/C#) suffices, TestComplete would be better for them, plus I can help write the scripts for them behind the scenes.  Take care and thanks again.