Forum Discussion

rwood_cert's avatar
rwood_cert
Occasional Contributor
2 years ago

TestComplete Test Adapter (v15.53.9) issues running tests in DevOps

Hi,

 

We're currently seeing issues running our tests via TestExecute, we believe this is being caused/coincided with the update to the TestComplete test adapter yesterday as our tests were completely fine on the evening of the 25th July, we run tests daily, with tests on the 26th July failing after the new version of the test adapter was released.

 

I've listed a snippet of the build log as below;

 

"2023-07-27T08:49:29.7207073Z ##[error]Failed to get a list of tests from the "C:\a\r1\a\<REPONAMERemoved>\<FileName>.pjs" file due to the following error: Cannot start process because a file name has not been provided.
2023-07-27T08:49:29.7214527Z No test matches the given testcase filter `Group=InstallationAndLicencing` in C:\a\r1\a\<REPONAMERemoved>\<FileName>.pjs
2023-07-27T08:49:29.7950327Z Results File: C:\a\_temp\TestResults\AzDevOps_UIMCUKP0002RD_2023-07-27_08_49_29.trx
2023-07-27T08:49:29.8173580Z ##[error]Test Run Failed."

 

In an attempt to rule out any issues our side, we've updated to the latest version of TestExecute that we can download, v15.53.2 and installed this on the VM image that gets installed on our VMSS, but not change.

 

Is there some other modification we're required to do? 

Or is there an issue within the test adapter within DevOps - https://marketplace.visualstudio.com/items?itemName=SmartBearSoftware.install-testcomplete-adapter-task&ssr=false#overview

 

Thanks

Test Run
  • PMularczyk's avatar
    PMularczyk
    2 years ago

    Hey,

     

    we just released new version of Test Adapter.

     

    Best regards,

    Pawel

  • quanderer_7's avatar
    quanderer_7
    Staff
    2 years ago

    Look like this is an issue with TE installation path, I installed TE to a different path, not in C:\Program Files (x86), got the same error, but if I installed TE using the default path C:\Program Files (x86)\SmartBear\TestExecute 15, then everything is working again. You guys can try to see if it works for you.

  • dfajardo's avatar
    dfajardo
    New Contributor
    2 years ago

    Try these task version numbers to revert back to a working state.  You can look what version number you used in your last successful run and use that version number or the ones I've provided.

     

    # Install VS Test Platform
    - task: VisualStudioTestPlatformInstaller@1
    displayName: Install VS Test
    inputs:
    packageFeedSelector: 'nugetOrg'
    versionSelector: 'latestStable'

    # Install TC Test Adaptor
    - task: SmartBearSoftware.install-testcomplete-adapter-task.install-testcomplete-adapter-task.InstallTestCompleteAdapter@1.15040.421
    displayName: Install TC Test Adaptor
    inputs:
    installExecutor: true
    updateExecutor: false
    logsLevel: '0'
    cmdLineParams: '/PrjVar:ExecutionRegion=${{ parameters.Region }}'
    preferredExecutor: TE

     

    # VS Test Run
    - task: VSTest@2.220.0
    displayName: VS Test Run
    inputs:
    testSelector: 'testAssemblies'
    testAssemblyVer2: '**\$(PJS_FILE)'
    searchFolder: '$(Build.SourcesDirectory)'
    testFiltercriteria: 'Name=${{ parameters.TestFilter }}'
    vsTestVersion: 'toolsInstaller'

  • SlickRick's avatar
    SlickRick
    Contributor
    2 years ago

    dfajardo :

    My pipeline is not using YAML yet and therefore not sure how i can force a version to be used.

     

    Here are the version im using:

    VSTest (Visual Studio Test😞

    Task : Visual Studio Test
    Version : 2.170.1
     
    TC Test adapter:
    Task : Enable support for TestComplete tests
    Version : 1.15053.9
     

    So for VSTest I seem to be using an older version than the one you are mentioning.

    But again, I don't know how to enforce a version without having to convert my pipeline to YAML which is not really what i desire at this stage.

    Anyways since im using TFVC im not even able to use a YAML pipeline.

    • dfajardo's avatar
      dfajardo
      New Contributor
      2 years ago

      Your classic pipeline created a file called azure-pipeline.yml in the repository you setup the pipeline with.  You can go to that file and directly change the version number as I pointed out.  WARNING: not sure if that might mess up your ability to use the classic editor.  But everything you do gets created as a azure-pipeline.yml file for classic pipelines.

      • SlickRick's avatar
        SlickRick
        Contributor
        2 years ago

        dfajardo I looked at the repository and this is not the case. No yml files are there. I believe this was the case a long time ago but it is no longer the case.

         

        I could try to manually update the pipeline using the REST api (we do this internally to update using script or pipeline whenever we need to propagate changes to our many pipelines).

         

        I'll let you know if that works. However clearly this is not a viable solution in long term, but if it works shortterm then great.

  • SlickRick's avatar
    SlickRick
    Contributor
    2 years ago

    dfajardo sadly not too sure if ill be able to do that. The export to YAML feature of TFS crashes with an 'Object reference not set to an instance of an object'. Im guessing it might be because im using TFVC and which may not be supported by YAML... 

     

    I currently trying to force the version, sadly it seems that for VSTEST it is not allowed and the REST API rejects anything except "2.*".

    For the TestComplete step, im able to enforce the version and the test seems to run. But as soon as i edit my pipeline, the version number is reenforced again to the latest one. So not very convenient. 




  • jerrymark's avatar
    jerrymark
    Occasional Visitor
    2 years ago

    You can use these general debugging steps to fix problems with the TestComplete Test Adapter in DevOps:

     

    Check Compatibility: Make sure that the version of TestComplete Test Adapter you are using is compatible with the version of DevOps you have loaded.

     

    Check Configuration: In DevOps, double-check your test configuration to make sure the Test Adapter is set up properly.

     

    Review Logs: Look for any problem messages or logs in DevOps, TestComplete, or the system where the tests are running that have to do with how the tests are being run.

     

    Update the TestComplete Test Adapter: Check to see if there are any updates or fixes for the TestComplete Test Adapter, and if there are, you might want to update it to the most recent version.

     

    Check if TestComplete works with the Test Adapter in DevOps. If you just updated TestComplete, make sure the version you have works with the Test Adapter.

     

    Check Your TestComplete Licence: Make sure your TestComplete licence is current and valid. An outdated licence can cause problems.

     

    Review Code and Tests: Look at your tests and code in TestComplete for any problems that could cause your tests to fail in DevOps.

     

    Check the Environment: Make sure that the DevOps test environment, which is where the tests are run, is properly set up and setup.spotify receiptify

     

  • nandini_thota's avatar
    nandini_thota
    Contributor
    2 years ago

    I tried today after the Test adapter version is released.

    Now, I am getting a different error.

    It says, TE should be updated. But I've checked the Install and Update Test runner options in the build.

    Am I missing out anything here?

     

  • PMularczyk's avatar
    PMularczyk
    Staff
    2 years ago

    Hey nandini_thota 

    please try this things:
    Try changing the preferred test runner here to, let's say, 32bit and back to what you had


    If that doesn't help, please check that the options.json (_work\1\s\TestCompleteAdapter) in the agent folder on which this is executed contains this "preferredRunner": "TE"

    If you have further problems, please write to our support. We will certainly be able to solve it then.

    Best regards,
    Pawel



    • nandini_thota's avatar
      nandini_thota
      Contributor
      2 years ago

      I didn't do any changes and it worked now. Previously, Test Execute didn't get installed properly. Now, When I triggered the job....it got installed and working now

  • rickyclerk's avatar
    rickyclerk
    New Member
    2 years ago

    here are some steps you can take to diagnose and potentially resolve the issues:

    1. Check Compatibility: Ensure that the version of TestComplete Test Adapter (15.53.9) you're using is compatible with the version of DevOps you're using. Compatibility issues can sometimes lead to unexpected behavior.

    2. Review Release Notes and Documentation: Consult the release notes and documentation for both TestComplete Test Adapter and your DevOps platform. There might be specific guidelines, known issues, or workarounds mentioned that could help you address the problems you're facing.krowd darden com