Forum Discussion

lydincamilleri's avatar
lydincamilleri
Occasional Contributor
3 years ago

Azure DevOps Integration - Re-running Failed Test

Hi Community,

 

Did anyone manage to find a way how to re-run failed tests using AzureDevops?

 

We currently are facing the problem that when a test fails inside a job (ex: a job has 5 tests, 4 passed but 1 failed), when rerunning failed jobs, all the 5 tests are re-executed. We would like to run only the specific test case which failed rather than the full job.

 

Appreciate your feedback :)!

Integrations
Script Tests
Test Results
Test Run
  • npaisley's avatar
    npaisley
    Staff
    3 years ago

    Hi lydincamilleri!

     

    One solution here would be to setup the 'On error' parameter for your tests to 'Stop and rerun current item'. This way whenever an error occurs TestComplete will rerun any tests that fail. You can even specify how many times you would like to rerun a failed test.

     

    This can be altered from the Execution plan for each test, or within the Project properties as well. 

     

    From Project properties;

     

    From the Execution Plan;

     

    Here is our document that outlines controlling your test flow within TC;

    https://support.smartbear.com/testcomplete/docs/testing-with/running/control-test-flow/overview.html

     

    I hope this helps!

     

  • lydincamilleri's avatar
    lydincamilleri
    Occasional Contributor
    3 years ago
    Thanks for your informative reply! I have tried that option but the way test items in our project are set up, the option was not sufficient.

    Fortunately, I have now managed to go around this issue using Azure DevOps VSTest's ability to re-run failed tests capped to a maximum retry count.

    Best regards,

    Lydin Camilleri
    Senior IT Quality Assurance Technical Analyst
    APS Bank plc
    Tel:
    +356 2560 3485
    Email:
    lydin.camilleri@apsbank.com.mt



    [cid:image002.png@01D87115.01343440]<>

    [cid:image003.jpg@01D87115.01343440]<>

    [cid:image004.jpg@01D87115.01343440]<>

    [cid:image005.jpg@01D87115.01343440]<>

    [cid:image006.jpg@01D87115.01343440]<>





    The information contained in this message and/or any of its attachments is intended solely for the exclusive use of the addressee(s) stated above and may be confidential and/or legally privileged. If you are not the addressee(s) you are hereby notified that any unauthorised disclosure, reproduction, copying, dissemination, distribution, use or taking any action in reliance on its contents is strictly prohibited and may be unlawful. If you have received this message in error, please delete it immediately from your system and destroy any hardcopies and kindly inform the author forthwith. APS Bank plc is neither liable for the proper and complete transmission of the information contained in this message and/or its attachments nor for any delay in its receipt. Furthermore, any opinions or views expressed in this message and/or its attachments belong to the author only, except where the message states otherwise and the sender is authorised to state them to be the views of APS Bank plc and/or any of its subsidiaries.
    Please consider your environmental responsibility before printing this email.
  • Yusufm87's avatar
    Yusufm87
    Contributor
    2 years ago

    Hi lydincamilleri,

     

    Could you please post your solution on how you were able to enable re-run of failed test cases.

    I am using Azure DevOps to run the tests but the failed test do not rerun at all.

     

    Thanks,

    Yusuf

    • lydincamilleri's avatar
      lydincamilleri
      Occasional Contributor
      2 years ago
      - task: VSTest@2
      inputs:
      testSelector: 'testAssemblies'
      testAssemblyVer2: '**\*.pjs'
      searchFolder: '$(System.DefaultWorkingDirectory)'
      distributionBatchType: 'basedOnExecutionTime'
      uiTests: true
      vsTestVersion: toolsInstaller
      diagnosticsEnabled: true
      collectDumpOn: 'always'
      rerunFailedTests: true
      rerunMaxAttempts: '2'
      testFiltercriteria: '@test=true'
      testRunTitle: TEST'
      displayName: 'TEST'