Forum Discussion

mhoult22's avatar
mhoult22
Occasional Contributor
2 years ago

TestExecute Warnings show as Errors in Azure Pipeline

Since updating TC and TE to Version: 15.58.9.7 x64 we are seeing issues running tests in our Azure release pipeline.

We have test scripts running using the TestComplete adaptor (TestExecute as "preferred test runner"), and Visual Studio Test.

If a test step completes with a Log.Warning, then now this is being treated by the pipeline as an error and is failing the test run.  To workaround this we have had to change all instances in our test code of Log.Warning to Log.Message. 

An example of a warning in the pipeline is:

##[error]RunMessage : Task 1_2: Modify list settings tab test: Exception occurred. (Exception from HRESULT: 0x80020009 (DISP_E_EXCEPTION))

If I look in TestComplete at the log of this "error" it is actually only a Warning, but the pipeline is reading it as an error.

This is new behavior since we updated TC and TE to the latest release.

Please could anyone help advise about this issue?

Thank you.

 

 

12 Replies

Sort By
  • fdering's avatar
    fdering
    New Contributor
    3 months ago

    The problem does not seem to be limited to TestComplete version 15.

    Since Monday the 17th of February, we have also been experiencing Azure release pipeline errors 
    HRESULT: 0x80020009 (DISP_E_EXCEPTION)) as described above.

    We are using TestExecute v14.93.312.11 and have not changed our environment.
    The exception is logged to the Azure release pipeline whenever we log a warning in our Python code using Log.Warning(...). 



  • fdering's avatar
    fdering
    New Contributor
    3 months ago

    Just found out that we had a different version of TestComplete TestAdapter before the problem was seen. On Monday Feb 17th we were using TestComplete TestAdapter v15.62 and had no problems.
    Later we were automatically updated to 15.72 and had problems.

    • scot1967's avatar
      scot1967
      Icon for Champion Level 1 rankChampion Level 1
      3 months ago

      I have seen several posts here related to 15.72. I don't know if this is related however.  I am glad you submitted a support case on this.  Let us know how it turns out!

      • fdering's avatar
        fdering
        New Contributor
        3 months ago

        SmartBeart just informed me that they released a hotfix 15.72.156 yesterday that should fix the issue. The previous version was 15.72.111.

  • Marsha_R's avatar
    Marsha_R
    Icon for Champion Level 3 rankChampion Level 3
    2 years ago

    I would report that directly to Support. Here's the link:

    https://support.smartbear.com/testcomplete/message/

    • mhoult22's avatar
      mhoult22
      Occasional Contributor
      2 years ago

      Thank you.  Case logged 00587108

      • Marsha_R's avatar
        Marsha_R
        Icon for Champion Level 3 rankChampion Level 3
        2 years ago

        Please post the answer here when you hear back. Thanks!

      • mhoult22's avatar
        mhoult22
        Occasional Contributor
        3 months ago

        Thanks for the link scot1967, I hadn't seen that issue.  I guess it could be related (certainly it won't help!), but we've had the Warnings as Errors issue for 2 years now, and the linked issue relates to the latest test adapter version.

        We've also got a support case raised with Smartbear for the new adapter (#00678070), as it is showing all test steps in a test plan as failed if only a single test step fails.  This makes ADOs results totally inaccurate.  Only if a test plan has a 100% pass rate does it show as passed, else it will show every single step as failed.

    • mhoult22's avatar
      mhoult22
      Occasional Contributor
      4 months ago

      Hi.  Our original support case was closed as we thought the issue was resolved, but it came back.  We have a new case: # 00691802.  It's status is: Waiting on SmartBear (Awaiting Fix)

      From some investigation it seems the reason we thought it was resolved was because of the "TestComplete test adapter installer" setting reportAnyError.

      In a test we did, with reportAnyError=false we see two errors. One correctly from Log.Error, but also one from Log.Warning, which is incorrect.

      With reportAnyError=true, the only "error" is from Log.Error, as expected, but none of the Log.Warnings show in the Azure pipeline, they are showing as "Passed", which is incorrect.

      So we waiting now to see if Smartbear have a fix.

  • fdering's avatar
    fdering
    New Contributor
    3 months ago

    I suspect that a certificate has expired, preventing a COM server from running.
    I have opened a support case at 00694992.