Forum Discussion

IAG's avatar
IAG
Occasional Contributor
7 years ago

Test Result analysis with automation tool : "first detection" Vs "already known" defects

Hello,  After execution of test case we must analyze the test result and check the defect detected by tool like Test Complete, tosca tricentis, selenium or other. In the first iteration, we can eas...
  • tristaanogre's avatar
    7 years ago

    Not within the TestComplete tool itself, no.  The tool is what it is: a development and execution tool for automated functional tests.  The output from the tool is aimed at simply reporting the results of a test run as executed.

     

    For test result analysis, that's what tools like QA Complete are for.  Investigate in that direction.

  • AlexKaras's avatar
    7 years ago

    Hi Ismail,

     

    One of the possible ways (exact implementation depends on the used tools/components):

    -- Prerequisite: a) issue tracking system (ITS) is used to register problems; and b) it is possible to query ITS via some means (DCOM, http, tcp, etc.);

    -- Within your common reusable test code, implement assert method that accepts condition (true/false) and issue identifier as parameters;

    -- Within this assert method, you should check the condition. If the condition is true, then everything is fine;

    -- If the condition fails, then check issue identifier. If it is empty (or zero, or null, ...) then this means that this is the 'first detection' that must be reported;

    -- After you analyse test log and decide that the reported problem is a problem indeed, you must report it to the ITS and provide its identifier in test code as a parameter to the relevant call to assert;

    -- On subsequent test runs, when condition fails again, you need to query the ITS for the issue status. If the status is not the final one (e.g. 'fixed/deployed'), then this means that the problem is known but not fixed yet. So you may report it as an 'already known' one;

    -- If the condition fails but the status of the issue is the final one, this means that this is 'regression' detection.