Forum Discussion

DenisMedvedev's avatar
DenisMedvedev
Contributor
8 years ago
Solved

How to deal with suit and sub-suit test items?

 I have Regression Suit and some sub-suits, see below:

 

 

 

The question is: How to stop running Suit 1 (Test 2 and Test 3) if Test 1 failed and continue to run Suit 2 and Suit 3.

I can add some logic in test scripts, but seems like we shoud have easier way.

 

I would be grateful for any help.

Thanks in advance

Denis

  • Montikore's avatar
    Montikore
    8 years ago

    imo, all tests in a test campaign are to be independant of each others... and this is kind of a standard. This is probably why you can't do this easily in TestComplete

  • Hi Denis,

     

    Unfortunately, this is not possible in TestComplete now.

    'Test Item' setting in the test items tree means that TestComplete will stop execution of the current test item, will skip execution of its children and continue with the test item of the same (or parent if no sibling test items are left) level.

    I would love to see a setting effectively meaning 'navigate to parent node and continue execution from there' but it does not exist as we speak.

    You may either implement own coded logic to make a decision of whether the given test item must be skipped or executed or consider tests to be organized, for example, as suggested by Colin (https://community.smartbear.com/t5/Functional-Web-Testing/Strategy-for-Disabling-Stop-on-Error/m-p/120537#M27617)

     

     

    P.S. Tests's independency mentioned by Montikore is a reasonable idea which, as they say, works with unit tests, but I never saw it to be (effectively) working in an extensive set of complex end-to-end functional tests...

  •  
     
     
     
    I am Brazilian, I implemented a solution for when there is failure and can continue. I hope I can spend my idea for you showing the code.
    If you have any doubts regarding abaxo I am willing to help you.

    But in short:

    I have three red variable standards that define where the test is;
    I have a procedure to define the test cases that MT will be executed;
    I used OnLogError to override the error, to know the mistake I used UserForms
    The variables are integer
    What do I identify the tests cases, I use the ComboBox ID to set the value of the variables thus being easy to continue (example) (

    UserForms.Error.cbSelBase.ItemIndex.Add ('CT_2_2');

    Summarize the maximum my test project.
    But let's say my TestExecute, does everything the TestComplete itself does less correct code. My goal: artificial intelligence (laughs) kkk.
     
    ----------------------------------------------------------------------------------------------------------------
     
    My test items

    CT_2_2 procedure;
    begin
           ShowMessage ( 'CT_2_2 test');
    end;
    Start Test
    procedure beginning;
    begin
          Principal.Etapa_1;
    end;

    -------------------------------------------------- ------------
    procedure CT_2_4;
    begin
          ShowMessage ( 'CT_2_4 test'); // If there is the will to error OnLogErro
    end;

    procedure CT_2_6;
    begin
            ShowMessage ( 'CT_2_6 test');
    end;

    procedure CT_2_8;
    begin
            ShowMessage ( 'CT_2_8 test');
    end;

    -------------------------------------------------- -------------------------------------------------

    Events create OnLogError event - It is only called when an error occurs, then you can use it to treat your TestItem the large balcony that creates vc test item in the 'code'

    GeneralEvents_OnLogError procedure (Sender; LogParams);
    var VerificouImagem: integer;
    ShowLogError: boolean;
    begin

    // Right now I implemented a form to handle the error
    // So having a button that setava my variable to continue the test
    // My test consists of three key variables: Base, Step and cases represented by Project.Variables.PrincipalBase, Project.Variables.PrincipalStep and Project.Variables.PrincipalCases



    // If this will only be able to continue the test if clicked Continue on ERROR form.
    if (Project.Variables.ContinuaTesteErro = true) then // is veriavel is assigned the button to confirm the Form Error
    begin
               Project.Variables.ContinuaTesteErro: = false;
               ContinuaTesteError; // Here determine the continuation of the test
    end;
    end;

    end;

    -----------------------------------------------------------------------------------------------------
    Specifies all tests

    ContinuaTesteError procedure;
    begin
        case Project.Variables.PrincipalStep of

           0: Principal.Etapa_1;
           1: Principal.Etapa_2;
           2: Principal.Etapa_3;
         end;
    end;
    ------------------------------------------------------------------------------------------------------
    Determines which IRAM tests run by stage

    procedure Etapa_1;
    begin

    if (Project.Variables.PrincipalCases= 0) then
    begin
          CT_2_2; // Call my Test
          Project.Variables.PrincipalCT Project.Variables.PrincipalCT + = 1; // Move to the next test
    end;

    if Project.Variables.PrincipalCases = 1) then
    begin
         CT_2_4;
         Project.Variables.PrincipalCT = Project.Variables.PrincipalCT + 1;
    end;

    if Project.Variables.PrincipalCases = 2) then
    begin
          CT_2_6;
          Project.Variables.PrincipalCT = Project.Variables.PrincipalCT + 1;
    end;

    if Project.Variables.PrincipalCases = 3) then
    begin
          CT_2_8;
          //Project.Variables.PrincipalCT = Project.Variables.PrincipalCT + 1;
    end;

    end;

    procedure Etapa_2;
    begin

    if Project.Variables.PrincipalCases = 0) then
    begin
            CT_2_22;
           Project.Variables.PrincipalCT = Project.Variables.PrincipalCT + 1;
    end;

    if Project.Variables.PrincipalCases = 1) then
    begin
            CT_2_44;
            Project.Variables.PrincipalCT = Project.Variables.PrincipalCT + 1;
    end;

    if Project.Variables.PrincipalCases = 2) then
    begin
            CT_2_66;
            Project.Variables.PrincipalCT = Project.Variables.PrincipalCT + 1;
    end;

    if Project.Variables.PrincipalCases = 3) then
    begin
            CT_2_88;
            //Project.Variables.PrincipalCT = Project.Variables.PrincipalCT + 1;
    end;

    end;

    procedure Etapa_3;
    begin

    if Project.Variables.PrincipalCases = 0) then
    begin
            CT_2_222;
            Project.Variables.PrincipalCT = Project.Variables.PrincipalCT + 1;
    end;

    if Project.Variables.PrincipalCases = 1) then
    begin
            CT_2_444;
            Project.Variables.PrincipalCT = Project.Variables.PrincipalCT + 1;
    end;

    if Project.Variables.PrincipalCases = 2) then
    begin
            CT_2_666;
            Project.Variables.PrincipalCT = Project.Variables.PrincipalCT + 1;
    end;

    if Project.Variables.PrincipalCases = 3) then
    begin
            CT_2_888;
            //Project.Variables.PrincipalCT = Project.Variables.PrincipalCT + 1;
    end;

    end;


    Thus becomes a Looping.
     

     

    I have helped thank you

14 Replies

    • Montikore's avatar
      Montikore
      Occasional Contributor

      create tests who are calling successively your sub-tests, with "stop on error" option activated.

      i'm not sure there is a "nice" way to do that...

  • AlexKaras's avatar
    AlexKaras
    Champion Level 3

    Hi Denis,

     

    Unfortunately, this is not possible in TestComplete now.

    'Test Item' setting in the test items tree means that TestComplete will stop execution of the current test item, will skip execution of its children and continue with the test item of the same (or parent if no sibling test items are left) level.

    I would love to see a setting effectively meaning 'navigate to parent node and continue execution from there' but it does not exist as we speak.

    You may either implement own coded logic to make a decision of whether the given test item must be skipped or executed or consider tests to be organized, for example, as suggested by Colin (https://community.smartbear.com/t5/Functional-Web-Testing/Strategy-for-Disabling-Stop-on-Error/m-p/120537#M27617)

     

     

    P.S. Tests's independency mentioned by Montikore is a reasonable idea which, as they say, works with unit tests, but I never saw it to be (effectively) working in an extensive set of complex end-to-end functional tests...

    • Montikore's avatar
      Montikore
      Occasional Contributor

      I'm a bit surprised by your comment Alex. I'm currently working on complex end-to-end independant functionnal tests, it's going pretty well. The only dependance they could have is a "configuration" test for a group of tests.

      When they were dependant (i'm not the first one to work on these), it was just impossible to get a relevant status for the test campaign, as one error could led to a lot of failed tests.

      Also, maintenance of your tests will be very difficult if you have to run a set of tests before being in the "good" state.

      So, the 2 main things for good automatic tests (reliability and easy maintenance) are not here...

       

      • p_bremm11's avatar
        p_bremm11
        Contributor
         
         
         
         
        I am Brazilian, I implemented a solution for when there is failure and can continue. I hope I can spend my idea for you showing the code.
        If you have any doubts regarding abaxo I am willing to help you.

        But in short:

        I have three red variable standards that define where the test is;
        I have a procedure to define the test cases that MT will be executed;
        I used OnLogError to override the error, to know the mistake I used UserForms
        The variables are integer
        What do I identify the tests cases, I use the ComboBox ID to set the value of the variables thus being easy to continue (example) (

        UserForms.Error.cbSelBase.ItemIndex.Add ('CT_2_2');

        Summarize the maximum my test project.
        But let's say my TestExecute, does everything the TestComplete itself does less correct code. My goal: artificial intelligence (laughs) kkk.
         
        ----------------------------------------------------------------------------------------------------------------
         
        My test items

        CT_2_2 procedure;
        begin
               ShowMessage ( 'CT_2_2 test');
        end;
        Start Test
        procedure beginning;
        begin
              Principal.Etapa_1;
        end;

        -------------------------------------------------- ------------
        procedure CT_2_4;
        begin
              ShowMessage ( 'CT_2_4 test'); // If there is the will to error OnLogErro
        end;

        procedure CT_2_6;
        begin
                ShowMessage ( 'CT_2_6 test');
        end;

        procedure CT_2_8;
        begin
                ShowMessage ( 'CT_2_8 test');
        end;

        -------------------------------------------------- -------------------------------------------------

        Events create OnLogError event - It is only called when an error occurs, then you can use it to treat your TestItem the large balcony that creates vc test item in the 'code'

        GeneralEvents_OnLogError procedure (Sender; LogParams);
        var VerificouImagem: integer;
        ShowLogError: boolean;
        begin

        // Right now I implemented a form to handle the error
        // So having a button that setava my variable to continue the test
        // My test consists of three key variables: Base, Step and cases represented by Project.Variables.PrincipalBase, Project.Variables.PrincipalStep and Project.Variables.PrincipalCases



        // If this will only be able to continue the test if clicked Continue on ERROR form.
        if (Project.Variables.ContinuaTesteErro = true) then // is veriavel is assigned the button to confirm the Form Error
        begin
                   Project.Variables.ContinuaTesteErro: = false;
                   ContinuaTesteError; // Here determine the continuation of the test
        end;
        end;

        end;

        -----------------------------------------------------------------------------------------------------
        Specifies all tests

        ContinuaTesteError procedure;
        begin
            case Project.Variables.PrincipalStep of

               0: Principal.Etapa_1;
               1: Principal.Etapa_2;
               2: Principal.Etapa_3;
             end;
        end;
        ------------------------------------------------------------------------------------------------------
        Determines which IRAM tests run by stage

        procedure Etapa_1;
        begin

        if (Project.Variables.PrincipalCases= 0) then
        begin
              CT_2_2; // Call my Test
              Project.Variables.PrincipalCT Project.Variables.PrincipalCT + = 1; // Move to the next test
        end;

        if Project.Variables.PrincipalCases = 1) then
        begin
             CT_2_4;
             Project.Variables.PrincipalCT = Project.Variables.PrincipalCT + 1;
        end;

        if Project.Variables.PrincipalCases = 2) then
        begin
              CT_2_6;
              Project.Variables.PrincipalCT = Project.Variables.PrincipalCT + 1;
        end;

        if Project.Variables.PrincipalCases = 3) then
        begin
              CT_2_8;
              //Project.Variables.PrincipalCT = Project.Variables.PrincipalCT + 1;
        end;

        end;

        procedure Etapa_2;
        begin

        if Project.Variables.PrincipalCases = 0) then
        begin
                CT_2_22;
               Project.Variables.PrincipalCT = Project.Variables.PrincipalCT + 1;
        end;

        if Project.Variables.PrincipalCases = 1) then
        begin
                CT_2_44;
                Project.Variables.PrincipalCT = Project.Variables.PrincipalCT + 1;
        end;

        if Project.Variables.PrincipalCases = 2) then
        begin
                CT_2_66;
                Project.Variables.PrincipalCT = Project.Variables.PrincipalCT + 1;
        end;

        if Project.Variables.PrincipalCases = 3) then
        begin
                CT_2_88;
                //Project.Variables.PrincipalCT = Project.Variables.PrincipalCT + 1;
        end;

        end;

        procedure Etapa_3;
        begin

        if Project.Variables.PrincipalCases = 0) then
        begin
                CT_2_222;
                Project.Variables.PrincipalCT = Project.Variables.PrincipalCT + 1;
        end;

        if Project.Variables.PrincipalCases = 1) then
        begin
                CT_2_444;
                Project.Variables.PrincipalCT = Project.Variables.PrincipalCT + 1;
        end;

        if Project.Variables.PrincipalCases = 2) then
        begin
                CT_2_666;
                Project.Variables.PrincipalCT = Project.Variables.PrincipalCT + 1;
        end;

        if Project.Variables.PrincipalCases = 3) then
        begin
                CT_2_888;
                //Project.Variables.PrincipalCT = Project.Variables.PrincipalCT + 1;
        end;

        end;


        Thus becomes a Looping.
         

         

        I have helped thank you