krkarthik_info's avatar
krkarthik_info
Frequent Contributor
6 years ago
Status:
New Idea

Requesting a feature to run only the failed test items in the project suite

Hi All,

 

I would like to have a feature in SmartBear TestComplete, where we can run only the failed test items in the project suite.

Explaining in detail:

Assume I have a project suite with 3 projects with that suite, and each project has around 100 test items to be executed. So in total 300 test items are to be executed, when execution triggered for project suite. After the first round of execution, assume in each projects 10 test items failed randomly.

Currently, what we have to do is, we need to select those failed test items manually and trigger execution again on the whole project suite. [This we have to do, when tests are failed due to some environment issues and so]

It would be good, if we have an option of running only the failed test items from the last run.

 

Thanks,

Karthik K R

  • tristaanogre's avatar
    tristaanogre
    Esteemed Contributor
    6 years ago

    What you are asking for is a test management tool.  TestComplete is the tool to use to create the tests.  To actually have that kind of control of test execution and re-execution, you can look into SmartBear's tool QAComplete or look at integrating with something like Zephyr or Jenkins to work on that kind of re-run process.

  • krkarthik_info's avatar
    krkarthik_info
    Frequent Contributor
    6 years ago

    Hi,

     

    Thanks for the reply.

     

    Actually we are using Jenkins to run TestComplete tests. In the end of execution, we consume the junitResult file generated by the TestComplete Test plugin and parse the results and publish that into the dashboard we maintain internally. 

     

    As you know, when we call the TestComplete Tests from Jenkins, it executes the test items associated either with the project suite or project. [Based on the selection Entire Suite or Project in TestComplete Test plugin in Jenkins]

     

    Assume my Test Item in one of the project is organized as below

    Regression
        |___
                UserStory1
                        |___
                               TestCase1
                               TestCase2

    Assume TestCase2 got failed on first run. In junitResult file generated, the entry will be as below:

    <case>
<duration>68.552</duration>
<className>Regression</className>
<testName>Regression/UserStory1/TestCase1</testName>
<skipped>false</skipped>
<errorStackTrace/>
<errorDetails>Error message</errorDetails>
<failedSince>2</failedSince>
</case>

    Our dashboard service parse the information based on testName tag and displays the execution result in the dashboard.

     

    We are highly dependent on junitResult file and the testName tag.

     

    I know we can rerun the failed test item, using Script Test option in TestComplete Test plugin, where we need to provide the specifiec project, unit and routine name.  But I was wondering post the execution, will the junitResult testName tag contains the same value as it was not part of test item in this case. If so, we cant use the dashboard parser to replace the existing failed test with the new result.

     

    So I was requesting for a feature, where TestComplete maintains some sort of xml files of all failed test items

     

    Thanks,

    Karthik K R

     

  • maxtester's avatar
    maxtester
    Contributor
    6 years ago

    We have solved the problem by writung a flat file with the result of each testcase (testiem). After rhe testrun we create (with powershell) a new .mds with only the testcases (testitems) which failed. Then we restart it automaticaly.