Forum Discussion

SlickRick's avatar
10 days ago

Azure Devops Release Pipeline with tags - Not running expected test

Hi all,

i'm trying to setup a release pipeline in AzureDevops that will run tests with a specific tag.

However when i run the release, it keeps on running all the tests and not just the one with the tags.

I'm trying to figure out what im doing wrong here...I

I tried to use both the /tags: and /test:  parameter none works...

Also i tried on the build machine to run it in command line and it works:
"C:\Program Files (x86)\SmartBear\TestExecute 15\x64\Bin\TestExecute.exe" "C:\.....\SESInstaller.UITestSuite.pjs" /run /project:"SESInstaller.UITests" /tags:"@DoNothingTest"


What am i doing wrong?

Thank you!


5 Replies

  • If this is true, this would be a huge deal breaker for us. My big plan is to be able to setup pipelines that runs various tests (@SmokeTest, @Sanity, @High, @Low, etc) at various stages of my CI/CD process. The advantage of using this direct via the Adapter and VSTEST is that you get log reports directly from the pipeline IDE.
    If i start using batch files and process run ill lose all that. 

    Yes i can export the log somewhere but i wont have that nice display. At least not without some big extra work...

  • rraghvani's avatar
    Champion Level 3

    If I remember correctly, I think tags are ignored via Azure DevOps (relating to the test adapter). The alternative way, is to call a batch file, with the command line that you have.

  • rraghvani's avatar
    Champion Level 3

    Search the TC forum for "DevOps Tags", there will be similar issues, and I don't think there's a solution.

    • SlickRick's avatar

      I'll do that. Thank you for replying to my post (and all my other posts :D).

      I've opened a ticket and hopefully there is a solution or at least maybe they will support it soon 

  • msensoy1903's avatar
    New Contributor

    Hello, I recently solve this issue with TestComplete. This is how your vstest-test assemblies supposed to look like in order for your tag to run.