Forum Discussion

SlickRick's avatar
SlickRick
Frequent Contributor
7 months ago

Azure Devops Release Pipeline with tags - Not running expected test

Hi all,

i'm trying to setup a release pipeline in AzureDevops that will run tests with a specific tag.


However when i run the release, it keeps on running all the tests and not just the one with the tags.

I'm trying to figure out what im doing wrong here...I

I tried to use both the /tags: and /test:  parameter none works...

Also i tried on the build machine to run it in command line and it works:
"C:\Program Files (x86)\SmartBear\TestExecute 15\x64\Bin\TestExecute.exe" "C:\.....\SESInstaller.UITestSuite.pjs" /run /project:"SESInstaller.UITests" /tags:"@DoNothingTest"

 

What am i doing wrong?

Thank you!

 

Keyword Tests
  • SlickRick's avatar
    SlickRick
    6 months ago

    So in order to do this you need to use the Test Filter field in the Visual Studio Test task with:
    @Sanity=true:

     

  • SlickRick's avatar
    SlickRick
    Frequent Contributor
    7 months ago

    If this is true, this would be a huge deal breaker for us. My big plan is to be able to setup pipelines that runs various tests (@SmokeTest, @Sanity, @High, @Low, etc) at various stages of my CI/CD process. The advantage of using this direct via the Adapter and VSTEST is that you get log reports directly from the pipeline IDE.
    If i start using batch files and process run ill lose all that. 

    Yes i can export the log somewhere but i wont have that nice display. At least not without some big extra work...

  • rraghvani's avatar
    rraghvani
    Champion Level 3
    7 months ago

    If I remember correctly, I think tags are ignored via Azure DevOps (relating to the test adapter). The alternative way, is to call a batch file, with the command line that you have.

  • rraghvani's avatar
    rraghvani
    Champion Level 3
    7 months ago

    Search the TC forum for "DevOps Tags", there will be similar issues, and I don't think there's a solution.

    • SlickRick's avatar
      SlickRick
      Frequent Contributor
      7 months ago

      I'll do that. Thank you for replying to my post (and all my other posts :D).

      I've opened a ticket and hopefully there is a solution or at least maybe they will support it soon 

  • msensoy1903's avatar
    msensoy1903
    Occasional Contributor
    7 months ago

    Hello, I recently solve this issue with TestComplete. This is how your vstest-test assemblies supposed to look like in order for your tag to run.

  • SlickRick's avatar
    SlickRick
    Frequent Contributor
    6 months ago

    So in order to do this you need to use the Test Filter field in the Visual Studio Test task with:
    @Sanity=true: