Forum Discussion

Florian_Hofmann's avatar
Florian_Hofmann
Contributor
8 years ago

What does TFS build step "Publish TestComplete test results" actually publish

Hi, let's say I want to start an application, perform 9 tests on it, and close the application again. This is implemented as 9 test items in the project suite, each associated with a testcase.   I...
  • Florian_Hofmann's avatar
    Florian_Hofmann
    8 years ago

    Well, sorry, I think I got unprecise towards weekend...yes, I did find the detailed test results, but kind of hidden deeply, and not in a way suitable for extracting daily reports at a glance.

     

    By the name of "publish Test Complete results", I just expected, and still would like, to have a TFS build not publish test results where each TC logFile is counted as one test, but where the TC test cases/items are summed up from within the logs (as one logFile documents multiple test cases, since I would not want to start and stop TestComplete between similar testCases at the same GUI element).

     

    But technically my question is answered as I now understand what is published, and I will see whether I find a way around it, or ask in the appropriate section if the behavior can be improved.

     

    And maybe my approach is not best practice...it's just based on the assumption that starting an application, and than adding an entry, renaming it, and finally deleting it are three testcases, 

    Naturally I would not want to start/close the application between testcases, and it seems awkward to start/stop TestComplete between each such testCase, just that each testVase gets its own logFile.

     

    Edit: Also, the log only gets attached for failed runs - if a run succeeds, as I see it, there's no way of telling hom many testCases in it succeeded.

Florian_Hofmann's avatar
Florian_Hofmann
Contributor

This leads me to what this screenshot shows....saying something like that this build does not contain any test results for the selected event (in the blue box on the bottom).

 

I was also hoping for an easy way of seeing how many automated tests have been run, and how many of them failed, when clicking on the build, since this is a number we evaluate daily.

Even if I managed to get to the log.mht as suggested on https://support.smartbear.com/testcomplete/docs/working-with/integration/ms-vs/alm/team-builds/running.html, it would not be optimal - if I included a 10th test case to the test run, it would only go into our reports if someone followed some click chain.

 

Edit: In case it matters: Since the build step "visual studio test" started throwing errors when aplied to the .tc12test VS item, I started using the build step "batch file", calling a batch file which in turn calls mstest on the .tc12test file

Florian_Hofmann's avatar
Florian_Hofmann
Contributor
8 years ago

Update: Now the link "detailed report" leads to a more populated view (Screenshot detailed_2), by double clicking the test run, I can get to the log (Screenshot log).

 

But this still does not allow us to evaluate as I hoped for, since it effectively is counting test runs rather than test cases.

  • tristaanogre's avatar
    tristaanogre
    Esteemed Contributor
    8 years ago

    In your Log.png, there's a "view detailed results" link.  That actually should take you to something like a TestComplete log

    • Florian_Hofmann's avatar
      Florian_Hofmann
      Contributor
      8 years ago

      Well, sorry, I think I got unprecise towards weekend...yes, I did find the detailed test results, but kind of hidden deeply, and not in a way suitable for extracting daily reports at a glance.

       

      By the name of "publish Test Complete results", I just expected, and still would like, to have a TFS build not publish test results where each TC logFile is counted as one test, but where the TC test cases/items are summed up from within the logs (as one logFile documents multiple test cases, since I would not want to start and stop TestComplete between similar testCases at the same GUI element).

       

      But technically my question is answered as I now understand what is published, and I will see whether I find a way around it, or ask in the appropriate section if the behavior can be improved.

       

      And maybe my approach is not best practice...it's just based on the assumption that starting an application, and than adding an entry, renaming it, and finally deleting it are three testcases, 

      Naturally I would not want to start/close the application between testcases, and it seems awkward to start/stop TestComplete between each such testCase, just that each testVase gets its own logFile.

       

      Edit: Also, the log only gets attached for failed runs - if a run succeeds, as I see it, there's no way of telling hom many testCases in it succeeded.