Forum Discussion

Contributor

6 months ago

Azure Devops Release Pipeline with tags - Not running expected test

Hi all,

i'm trying to setup a release pipeline in AzureDevops that will run tests with a specific tag.

However when i run the release, it keeps on running all the tests and not just the one with the tags.

I'm trying to figure out what im doing wrong here...I

I tried to use both the /tags: and /test: parameter none works...

Also i tried on the build machine to run it in command line and it works:
"C:\Program Files (x86)\SmartBear\TestExecute 15\x64\Bin\TestExecute.exe" "C:\.....\SESInstaller.UITestSuite.pjs" /run /project:"SESInstaller.UITests" /tags:"@DoNothingTest"

What am i doing wrong?

Thank you!

Keyword Tests

SlickRick
5 months ago
So in order to do this you need to use the Test Filter field in the Visual Studio Test task with:
@Sanity=true:

SlickRick
Contributor
6 months ago
If this is true, this would be a huge deal breaker for us. My big plan is to be able to setup pipelines that runs various tests (@SmokeTest, @Sanity, @High, @Low, etc) at various stages of my CI/CD process. The advantage of using this direct via the Adapter and VSTEST is that you get log reports directly from the pipeline IDE.
If i start using batch files and process run ill lose all that.
Yes i can export the log somewhere but i wont have that nice display. At least not without some big extra work...
rraghvani
Champion Level 3
6 months ago
If I remember correctly, I think tags are ignored via Azure DevOps (relating to the test adapter). The alternative way, is to call a batch file, with the command line that you have.
rraghvani
Champion Level 3
6 months ago
Search the TC forum for "DevOps Tags", there will be similar issues, and I don't think there's a solution.
- SlickRick
  Contributor
  6 months ago
  I'll do that. Thank you for replying to my post (and all my other posts :D).
  
  I've opened a ticket and hopefully there is a solution or at least maybe they will support it soon
msensoy1903
Occasional Contributor
6 months ago
Hello, I recently solve this issue with TestComplete. This is how your vstest-test assemblies supposed to look like in order for your tag to run.
SlickRick
Contributor
5 months ago
So in order to do this you need to use the Test Filter field in the Visual Studio Test task with:
@Sanity=true: