mhoult22
Occasional Contributor
Joined 2 years ago
User Profile
User Widgets
Contributions
Re: TestExecute Warnings show as Errors in Azure Pipeline
Thanks for the link scot1967, I hadn't seen that issue. I guess it could be related (certainly it won't help!), but we've had the Warnings as Errors issue for 2 years now, and the linked issue relates to the latest test adapter version. We've also got a support case raised with Smartbear for the new adapter (#00678070), as it is showing all test steps in a test plan as failed if only a single test step fails. This makes ADOs results totally inaccurate. Only if a test plan has a 100% pass rate does it show as passed, else it will show every single step as failed.100Views0likes0CommentsRe: TestExecute Warnings show as Errors in Azure Pipeline
Hi. Our original support case was closed as we thought the issue was resolved, but it came back. We have a new case: # 00691802. It's status is: Waiting on SmartBear (Awaiting Fix) From some investigation it seems the reason we thought it was resolved was because of the "TestComplete test adapter installer" setting reportAnyError. In a test we did, with reportAnyError=false we see two errors. One correctly from Log.Error, but also one from Log.Warning, which is incorrect. With reportAnyError=true, the only "error" is from Log.Error, as expected, but none of the Log.Warnings show in the Azure pipeline, they are showing as "Passed", which is incorrect. So we waiting now to see if Smartbear have a fix.111Views1like0CommentsTest steps get disabled automatically when a TestExecute test run fails in Azure pipeline
We have test scripts running in an Azure release pipeline, using the TestComplete adaptor (TestExecute as "preferred test runner"), and Visual Studio Test. We do NOT have the "Rerun failed tests" option in the Visiual Studio Test task enabled. If I have a test script, with all steps enabled in the execution plan, and any of these steps fail - when I log onto the agent virtual machine that has run the tests, I can see that the Execution Plan has all the passed test steps disabled and the failed test steps enabled. I've searched everywhere to find what controls this behavior, but cannot find anything. If a test fails I do not want any changes to the Execution Plan to be made automatically. Please could anyone help advise about this issue? Thank you.31Views0likes0CommentsTestExecute Warnings show as Errors in Azure Pipeline
Since updating TC and TE to Version: 15.58.9.7 x64 we are seeing issues running tests in our Azure release pipeline. We have test scripts running using the TestComplete adaptor (TestExecute as "preferred test runner"), and Visual Studio Test. If a test step completes with a Log.Warning, then now this is being treated by the pipeline as an error and is failing the test run. To workaround this we have had to change all instances in our test code of Log.Warning to Log.Message. An example of a warning in the pipeline is: ##[error]RunMessage : Task 1_2: Modify list settings tab test: Exception occurred. (Exception from HRESULT: 0x80020009 (DISP_E_EXCEPTION)) If I look in TestComplete at the log of this "error" it is actually only a Warning, but the pipeline is reading it as an error. This is new behavior since we updated TC and TE to the latest release. Please could anyone help advise about this issue? Thank you.342Views0likes12CommentsRe: Azure pipeline issue combining Test filter criteria in Visual Studo Test extension
Thanks rraghvani. Yes I had seen that page and was using it as a guide to try and help. In the end I dropped the "Project" criteria as I realised I didn't need it. I tried Enabled=true&Group!=Backup which didn't work, so I swapped them round to Group!=Backup&Enabled=true and this did work! Nothing mentioned that the order was important but it looks like it is.886Views0likes0CommentsAzure pipeline issue combining Test filter criteria in Visual Studo Test extension
Hi. I am having an issue with the Test filter criteria option within the Visual Studio Test extension in ADO. I can use this: Project=Preparation&Enabled=true and it works fine. I can use this: Group!=Backup which also works fine. But when I try to combine them both ADO seems to fail on the filter and does not run any tests at all. I have tried a lot of different combinations, but really I expected this to work but it doesn't: Project=Preparation&Enabled=true&Group!=Backup Any help would be appreciated. Thanks, MichaelSolved