Forum Discussion

ajohnson2020's avatar
ajohnson2020
Contributor
12 years ago

Capture pass/fail results at end of test run?

Hello everyone,



I was wondering if anyone has come up with a way to capture/publish just the pass/fail result counts from TestComplete at the end of each test run?  

I'm already publishing the logs in mht format, but they're over 20mb each unzipped (and that's with very few screenshots), so they're unwieldy for doing quick comparisons.  I would like to capture just the top level numbers for a lightweight dashboard.  I know QA Complete can do this, but I can't justify the expense just for this one piece of functionality.



Thanks.

Allen

5 Replies

  • chrisb's avatar
    chrisb
    Regular Contributor
    Hi Allen,



    I solved the problem of sharing large .mht files the following way:



    1. Create project suite variables to store counts of various test run events such as test starting, test stopping because of error, warning message logged. You can increment these when a event such as OnLogError occurs or from within scripts. I also write some data from theTestItems object  into the project suite variables such as the name of the failing test script.



    2. Add OnStopTest event to project.



    3. Write script for OnStopTest event that takes the counts stored in the project suite variables and formats them into a nice message containing test run metrics (number of tests in run, number of tests failed, number of warning messages, names of the failing test scripts.

    Use SaveResultsAs to write the .mht file to a network drive location and give it a unique filename (I use a mix of current date and time to generate the filename).

    Use SendMail to send an email notification to others on your project that includes test run metrics and the UNC path of the .mht file. 



    As a QA Complete user I think you are on the right track for not using QA Complete just for this one piece of functionility. 





  • karkadil's avatar
    karkadil
    Valued Contributor
    You can use Log.ErrCount property to get the number of errors in the current test.
  • hlalumiere's avatar
    hlalumiere
    Regular Contributor
    Or, do like I did and pipe all your test result summaries into a database, and extract your metrics from the database.
  • tristaanogre's avatar
    tristaanogre
    Esteemed Contributor
    What Chris B describes is what I've done in the past as well with the extra twist in that, if the Error count is 0 and the Warning count is below a particular threshhold, don't even bother exporting the log.  Basically the same rule of thumb when it comes to logging.  Don't write it out unless you need it.

  • Thanks everyone!  Many good responses and I'll probably try some variant of all of them.


    At least I definitely know I can do it now, I'll experiment for a while and see what works best for me.



    Allen