Forum Discussion

Southpaw's avatar
Southpaw
Contributor
5 years ago

TFS - Distributed Testing on a Remote Machine w TestExecute

Hello,

   I'm hoping to get some help with my build definition tasks in TFS (2015) to get TC tests to run using Test Execute on remote hosts. I have successfully created a Machine Group in TFS and no failures using the Visual Studio Test Agent Deployment task. I assume this is one of the steps I need to use for remote host testing out of TFS. But the Visual Studio Test task is still trying to execute the test(s) locally. See screen shot for my task steps. I think i need another step but just dont know which one. 

Thanks

9 Replies

Sort By
  • Southpaw's avatar
    Southpaw
    Contributor
    5 years ago

    Hello - 

       Ok I have found some progress. Please Note we are using TFS 2015. The Build Def Tasks Im using currently is:

    1) Visual Studio Build

    2) Windows Machine File Copy

    3) Visual Studio Test Agent Deployment

    4) Run Functional Tests

     

    Ive made a connection to our remote host pool. I am able to copy TesComplete project files to a network share. VS Test Agent deployment and configuration succeeds. But the Run Functional Tests begins but ultimately fails with the error 

    DistributedTests: Error : No tests were discovered from the specified test sources
    This is odd because I have all of the TC Project Files copied to the network share and all tasks map to this share. So something is wrong and I cant figure it out. ARE THERE ANY SmartBear Customers that actually use TC Integrated with VS and subsequently use TFS to run tests on distributed machines???
    This is beyond frustrating because there is a mountain of info to set up a distributed test run for Selenium, etc.. and I find nothing regarding TC. SmartBear docs say nothing about this approach. 
    Thanks.
    • RRostovtsev's avatar
      RRostovtsev
      SmartBear Alumni (Retired)
      5 years ago

      Hi Southpaw,

       

      There should not be any difference between distributed run of TC tests and standard tests, except that TestComplete test adapter should be installed on remote machines. You should also understand that *.tcTest (or *.tc12Test) files are actually considered as tests by TFS, not *.pjs.

      As far as I understand, this error

      No tests were discovered from the specified test sources

      can occur in two cases:

      1) TestComplete test adapter is not installed on a remote machine (Have you installed Visual Studio integration or Visual Studio Test Agent integration on your remote machines?)

      2) Test files are not found in the "Drop location folder" (Are you sure that *.tc*Test files are present in your \\ussdvucsd107 share?)

      Please also note that Microsoft marked "Run Functional Tests" step as deprecated and recommends to use "Visual Studio Test" step instead in next versions of TFS. And it is a good practice to store tests in a source control.

       

      Best regards,

      Roman

      • Southpaw's avatar
        Southpaw
        Contributor
        5 years ago

        When you say TestComplete test adapter - what exactly is this? Ive installed TestExecute on the remote machine that I am testing out these features on. I also have the VS Integration installed on the remote machine. 

        I'm aware of the deprecated task. I do not know what else to use as this task allows me to enter a machine group that contains the remote machine for this effort. The Visual Studio Test task doesnt offer that and when I run my build it attempts to run the test(s) on the build machine. 

        If you know the task when on TFS2015 to get the test to actually kick off on the remote machine(s) please let me know. 

         

    • JuliaBernikova's avatar
      JuliaBernikova
      SmartBear Alumni (Retired)
      5 years ago

      Hello,

       

      Thank you for posting your question here. The good news is that the approach you use is the one recommended in the TestComplete documentation (see the "Run TestComplete Tests as Part of Build Pipeline (Team Builds)" article). So, the steps you used in the build definition are absolutely correct.

       

      As for the "No tests were discovered from the specified test sources" error: you mentioned that the TestComplete project files had been copied to the specified location. However, the "Run Functional Tests" task can only run Visual Studio tests, not TestComplete projects. So, the agent is looking not for TestComplete project files (.pjs or .mds), but for TestComplete Visual Studio test files that reference actual TestComplete projects. These TestComplete Visual Studio tests files are the .tcTest or .tc12Test files. They are added to the Visual Studio solution and work as mediators between Visual Studio and TestComplete. The "Run Functional Tests" task tries to find these files by the mask specified in the "Test Assembly" field ("**\*.tc*Test"). So, first of all, please make sure that the location (\\ussdvucsd107\downloads\TestComplete) does indeed contain the .tcTest or .tc12Test files the agent is looking for.

       

      If the location only contains TestComplete projects, please follow the "2. Prepare Visual Studio Test Projects for Running TestComplete Tests (Run TestComplete Tests as Part of Build Pipeline)" and "3. Define a Build Process (Run TestComplete Tests as Part of Build Pipeline)" articles to make sure the solution is prepared and copied to the network location correctly.

       

      If the location does actually contain .tcTest or .tc12Test files, please make sure that the TestComplete integration package is installed on the test host. To see the entire list of requirements, please refer to "Run TestComplete Tests as Part of Build Pipeline - Requirements".

       

      Please also note that this approach is only recommended if you want to fully incorporate TestComplete tests into your TFS workflow and run them as usual Visual Studio tests. The approach might indeed be somewhat confusing as it involves a lot of steps to "trick" Visual Studio into thinking TestComplete tests are Visual Studio tests. But if your goal is to simply run TestComplete tests from TFS without any special reporting about the results, you can follow much simpler steps. Just copy the TestComplete projects to the test host, compose the command line necessary to run them (see "TestComplete Command Line") and execute this command line by using the "Command line" TFS task. Remember to make the TFS test agent interactive - that's required no matter the approach.