Forum Discussion

lherry's avatar
lherry
Contributor
2 years ago

Runner.Stop(true) coded in OnStartTestCase() event open a "test item failed to run" dialog box

Hello everyone,

In a context of "dynamically" selected tests coded in javascript, I would like to condition the execution of my tests items according to a defined list I have in memory.

 

I don't want to use the "enable/disable" checkbox of the test item because the state can't be changed during execution, so all my test items all selected...

I also don't want to generate an error to move to the next test (to avoid the error mark in the logs as it is not an error but a non-selection).

 

I was thinking of using the Runner.Stop(true) command which seemed to meet my need. This command works well as long as it is directly in the test script, however, in order not to have to copy the condition in each script, I would like to place it in the OnStartTestCase() event allowing to shunt the test before it even runs.

 

In the latter context, the command does the right thing but displays a dialog box asking for confirmation to continue after the test fails to run (both in TestComplete and TestExecute).

 

Does anyone have an idea or a solution to allow me to get around this problem without having to place this code directly in the test script?

 

Thanks in advance for your help,
Loïc

Desktop Testing
Mobile Testing
Script Tests
Test Creation
Test Results
Test Run
web testing
  • rraghvani's avatar
    rraghvani
    Champion Level 3
    2 years ago

    There doesn't seem to be OnStartTestItem() method in the Event Handler?

     

    I'm struggling to understand what you mean by "dynamically selected tests coded in javascript" and "defined list I have in memory."?

    • lherry's avatar
      lherry
      Contributor
      2 years ago

      You are right: it is not OnStartTestItem() but OnStartTestCase()... Sorry for the mistake 😁

  • lherry's avatar
    lherry
    Contributor
    2 years ago

    By "dynamic" I mean passing a configuration file to the project which indicates the test items selected or ignored via an internal business-oriented tool controlled by our project management.

     

    This configuration is interpreted by an dedicated javascript object which stores the list of tests selected in a field. It can then be queried to determine whether the current test should be run or skipped.

     

    Does this make more sense to you?

  • rraghvani's avatar
    rraghvani
    Champion Level 3
    2 years ago

    If you were to have Runner.Stop() in OnStartTestCase(), I haven't tried it myself, but I think it will just stop the test execution without running the test case!

     

    Also, if I remember correctly OnStartTestCase() will not be triggered if running tests from Azure DevOps.

     

    You could create a project suite, with projects, and write simple tests that just outputs to the log, with  the appropriate event handlers. Run the scripts either individually or using the execution plan, to see the behaviour of how TC runs the automation. Then do the same via your internal tool to see if you get the same behaviour as TC.

     

     

    • lherry's avatar
      lherry
      Contributor
      2 years ago

      Hello rraghvani,

       

      Thanks for the answer,

       

      First of all, just to make sure we understand each other, our internal tool is just a "remote control" to generate a configuration file that will be read by the scripts I coded and that are interpreted by TestComplete. So the test execution is always done by TestComplete, and never by our tool.

       

      Secondly, my concern is not with the running of the events but with the behaviour of the Runner.Stop(true) command when it is interpreted in the OnStartTestCase() event.

       

      This command is supposed to just stop the execution of the current test and it works fine if interpreted in the function associated with the test.

      However, when interpreted in the OnStartTestCase() event, it behaves differently by displaying a dialog box indicating that the current test has failed to run and asking whether or not to continue, thus blocking the execution of the project's test suite (similar behaviour to TestExecute).

       

      Hence my original post : is there any way to avoid this blocking behaviour?

       

      Thanks in advance for your help !

      • lherry's avatar
        lherry
        Contributor
        2 years ago

        A little additional information: if I answer that I want to continue executing the tests in the dialog box that appears, the behaviour is good: the execution goes on to the next test without executing the ignored test by marking it as OK.
        The problem is just the appearance of this dialog box...

  • rraghvani's avatar
    rraghvani
    Champion Level 3
    2 years ago

    Thanks for the clarifying. I hope this answers your question - to avoid this message appearing when using Runner.Stop(true)

    disable the following option, in your project settings

     

    • lherry's avatar
      lherry
      Contributor
      2 years ago

      I'm sorry, but after a few tests, it's not the solution because I don't have the dialog box anymore, but the tests are definitely interrupted, preventing the execution of the following tests.

      This behavior is consistent with what the dialog box indicated (exception not handled), which confirms that the management of this command is not fully functional in this context.

      I just opened a support case, we'll see.

       
  • lherry's avatar
    lherry
    Contributor
    2 years ago

    Thank you very much for your responsiveness!

    It unblocks me!

    I still wonder about the reason and the usefulness of this difference in behaviour depending on whether you use this function in the event or in the test function itself...

    Because, for me, using this command in the OnStartTestCase() event method should not trigger an exception.

    And so, with this option unchecked, I risk hiding any "real" exceptions.

  • rraghvani's avatar
    rraghvani
    Champion Level 3
    2 years ago

    TestComplete gives the option of handling exception manually using try...catch statement or allow TestComplete to deal with the exception via notification dialog. This is similar to most development applications.

    • lherry's avatar
      lherry
      Contributor
      2 years ago

      I tried the try/catch statement, but there is no exception raised at this level.
      I think it is thrown at a parent level...

      Also, the craziest thing is that my TestItem has its "On exception" property set to "Stop current item", which should allow (after unchecking the option you gave me before) to continue to the next test, which is not the case.

      In short, I'm in a technical bind at the moment.

  • rraghvani's avatar
    rraghvani
    Champion Level 3
    2 years ago

    I've set up TC so that I handle errors manually. In this example, I throw an error which is then caught, and then any final statement(s) is processed.

    However, in this example I want to stop any further processing once the exception is caught.

    You need to decide how you want your process to behave after an exception, either continue to run or gracefully exit.

    • lherry's avatar
      lherry
      Contributor
      2 years ago

      Thank you for your example

      This is exactly what I did, and I confirm that no exception is raised in the OnStartTestCase() method after the call of the Runner.Stop(true) command: no passage in the catch block.

      As the thrown exception prevents the execution of the test function and the OnStopTestCase() event function, it is impossible to perform a "clean" (or error handled) processing, leading to the blocking dialog box...

      SmartBear support is working on this. I'm waiting for them to get back to me, but it looks like a fix to be consistent with the documentation in this context.