Forum Discussion

kitaitoa's avatar
kitaitoa
Contributor
3 years ago

testexecute - pipeline Azure DevOps doesn't stop project on item

Hi,

I have searched a solution about my problem, but I didn't find anything. I hop somebody have a solution here 😉 : 


I have in my project 2 items with option "Stop project" on error.
when I execute it in testComplete it's ok :
If the first item is failed, the second items not run
, but in a Azure pipeline Yaml Azure DevOps after an error in the first item, the second item run also
- task: VSTest@2
displayName : "Tests Run : Basic Order KeywordTests"
condition: eq('${{ parameters.typedemo }}', 'VM-Basic Order KeywordTests -> Tests OK')
inputs:
testSelector: 'testAssemblies'
testAssemblyVer2: 'Desktop\Orders\Delphi\TCProjects\JavaScript\*.pjs'
searchFolder: '$(System.DefaultWorkingDirectory)'
testFiltercriteria: 'Project=Orders_Delphi_JavaScript'
runSettingsFile: 'RunSetting/test.runsettings'

I try to add an event on error but it's the same result
It seems the both items are distinct and not in the same project ..???
I share woth you my project zipped, contains too log and trx from the execution of my pipeline AZDO

Thanks for your help

Integrations
  • kitaitoa's avatar
    kitaitoa
    Contributor
    3 years ago

    Support SB tell me:

     When you are running tests in Azure, you are not running your full project or any other executable entity as you would in Jenkins or via CLI.
     
    Azure asks our adapter what tests are provided by a .pjs file and then runs them as it wants. Different tests may easily be executed on different machines should you provide multiple agents.
     
    In other words, the on-error behavior is controlled by Azure, not by TestComplete.

     

    but I dont understand this "solution" TC has an option that we can't used !
    Do you know if  this is possible in jenkins ?

  • Kitt's avatar
    Kitt
    Regular Contributor
    2 years ago

    A little late in responding, but wanted to provide a solution in case anyone else has a similar issue. The most common scenario is to stop the entire test run or prevent it from executing if the first test fails...

     

    Since Azure ignores the onLogError attempts to force Runner.Stop(), the simple solution would be to break this 1st test into it's own pipeline task. Then the subsequent task that runs the remaining tests can be configured to execute only when all previous tasks have succeeded. 

    • kitaitoa's avatar
      kitaitoa
      Contributor
      2 years ago

      Thank you Kitt for you reply

       

      I will try your solution, simple but great idea 😉

       

      See you

    • AlexKaras's avatar
      AlexKaras
      Champion Level 3
      2 years ago

      Another possible approach is to not use test adapter but start TestExecute via command line task. In this case TestExecute will have full control over project's Execution Plan and will be able to honor its OnError/OnException settings.

       

  • Kitt's avatar
    Kitt
    Regular Contributor
    3 years ago

    Since you are using the test assemblies task in your pipeline, have you set your 'on error' settings in both the project setting and execution plan (see [here])? You can add a line to log the exit code in your OnError event handler to see how TC exits your test when it fails (see [exit codes]).

    • kitaitoa's avatar
      kitaitoa
      Contributor
      3 years ago

      I'm not sure if your solution can help me... When I have the error code, how can I use it ?

  • John532's avatar
    John532
    New Contributor
    3 years ago

    I am also encountering this issue. Don't know why its happening.