Ask a Question

Azure Pipeline Does Not Run Selected Tests


Azure Pipeline Does Not Run Selected Tests

Hello everyone,


There are several moving parts that may be affecting this but I will try to keep it as simple as I can.


I created a basic keyword test that launches and closes Chrome. The test was added to an Execution Item in the Execution Plan and was assigned a tag for execution selection.

When I run the pipeline the tasks: Visual Studio Test Platform installer, TestComplete test adapter installer and VsTest - testAssemblies complete the execution successfully, although the expected script was not run.


The run log is not clear. It reads:

"A total of 1 test files matched the specified pattern.
The following test cases were skipped because they do not match the specified filter: AzureTest
No test matches the given testcase filter `@azure` in..."
even though I do have that tag in place in the keyword script.


I am using a virtual machine with TestExecute to run the pipeline and I noticed that files were temporarily created while the pipeline was running. I never got to see a Chrome browser opening or anything happening. I was expecting the notification tray to show the TestExecute agent to display, but I didn't see that either.


Any suggestions?



Community Hero

Does the basic keyword test run properly from TestComplete directly?  If so, then will it run with TestExecute from the command line correctly?

[Community Hero]
[Community Heroes] are not employed by SmartBear Software but
are just volunteers who have some experience with the tools by SmartBear Software
and a desire to help others. Posts made by [Community Heroes]
may differ from the official policies of SmartBear Software and should be treated
as the own private opinion of their authors and under no circumstances as an
official answer from SmartBear Software.
The [Community Hero] signature is used with permission by SmartBear Software.
Regular Contributor

What happens if you use TestComplete as your preferred test runner?



Assuming you have followed the TestComplete tagging reference [here] ? You can test locally on the machine the pipeline run on by opening up command-line and using something like this to test: TestComplete.exe project-suite-file /project:project-name /test:"@azure" /run


[[ kITt: keep IT testing ]]

Hello... sorry this is coming until now.


Just for reference for whoever encounters a similar issue. The root problem was the user with which the Azure agent was setup. Initially, the Azure agent was setup using the credentials of a member of the Infrastructure team. Then, I would log in with another user account which also had admin privileges. I was under the impression that this would make services available as they were for the user who setup the agent, but that was not the case. The Azure listener was conflicting and not launching automatically when I logged in with the user that would run the scripts. This also, had a side-effect on the TestExecute execution, etc.


The solution was to rerun the Azure agent setup, this time with the user that would be executing the tests and things look better now. Another related hint is that in the Visual Studio task I was including the "@" symbol as part of the filter, and that is not supposed to be included.


Thanks all for putting some time into finding a solution.





Showing results for 
Search instead for 
Did you mean: