Forum Discussion

Florian_Hofmann's avatar
Florian_Hofmann
Contributor
8 years ago

What does TFS build step "Publish TestComplete test results" actually publish

Hi,

let's say I want to start an application, perform 9 tests on it, and close the application again. This is implemented as 9 test items in the project suite, each associated with a testcase.

 

I integrated TC into visual studio, and when I run the tests, I get an .trx result file. When I double click it, it opens in VS and displays the results as they would be shown in TC:

9 Test items, each with a result and the possibility to open the tree structure to get information about each single step, comment and so on (attached file VS_Result)

 

 

 

When in Team Foundation Server I perform the build step "Publish TestComplete test results", it looks like this - only one test and no possibility to get any information about execution of test items.  (attached file TFS_Result)

I was hoping to see...well, actual TestComplete results in TFS...how would I do this?

 

Thanks in advance,

Florian

  • Florian_Hofmann's avatar
    Florian_Hofmann
    8 years ago

    Well, sorry, I think I got unprecise towards weekend...yes, I did find the detailed test results, but kind of hidden deeply, and not in a way suitable for extracting daily reports at a glance.

     

    By the name of "publish Test Complete results", I just expected, and still would like, to have a TFS build not publish test results where each TC logFile is counted as one test, but where the TC test cases/items are summed up from within the logs (as one logFile documents multiple test cases, since I would not want to start and stop TestComplete between similar testCases at the same GUI element).

     

    But technically my question is answered as I now understand what is published, and I will see whether I find a way around it, or ask in the appropriate section if the behavior can be improved.

     

    And maybe my approach is not best practice...it's just based on the assumption that starting an application, and than adding an entry, renaming it, and finally deleting it are three testcases, 

    Naturally I would not want to start/close the application between testcases, and it seems awkward to start/stop TestComplete between each such testCase, just that each testVase gets its own logFile.

     

    Edit: Also, the log only gets attached for failed runs - if a run succeeds, as I see it, there's no way of telling hom many testCases in it succeeded.

  • tristaanogre's avatar
    tristaanogre
    Esteemed Contributor
    8 years ago

    There's a link that, if Google Translate is correct, says "Detailed Report".  What happens when you click on that?

    • Florian_Hofmann's avatar
      Florian_Hofmann
      Contributor
      8 years ago

      This leads me to what this screenshot shows....saying something like that this build does not contain any test results for the selected event (in the blue box on the bottom).

       

      I was also hoping for an easy way of seeing how many automated tests have been run, and how many of them failed, when clicking on the build, since this is a number we evaluate daily.

      Even if I managed to get to the log.mht as suggested on https://support.smartbear.com/testcomplete/docs/working-with/integration/ms-vs/alm/team-builds/running.html, it would not be optimal - if I included a 10th test case to the test run, it would only go into our reports if someone followed some click chain.

       

      Edit: In case it matters: Since the build step "visual studio test" started throwing errors when aplied to the .tc12test VS item, I started using the build step "batch file", calling a batch file which in turn calls mstest on the .tc12test file

      • Florian_Hofmann's avatar
        Florian_Hofmann
        Contributor
        8 years ago

        Update: Now the link "detailed report" leads to a more populated view (Screenshot detailed_2), by double clicking the test run, I can get to the log (Screenshot log).

         

        But this still does not allow us to evaluate as I hoped for, since it effectively is counting test runs rather than test cases.