tphillips's avatar
tphillips
Frequent Contributor
4 years ago
Status:
New Idea

Keyword test clean-up/post-conditions

It would useful to be able to define clean-up steps/post-conditions for keyword tests. This could be implemented as a new test step section at the bottom that always gets called regardless of whether the main test failed or not.

The test would still be reported as having failed in the test results, but before moving onto the next test it calls the clean up steps.

 

I know that you could implement this as a separate test that gets called after the main test in the test project, but that's a pain if multiple tests have the same clean-up steps. Having explicit clean-up steps makes it easier to understand what the test does just by looking at it, and how it might affect the rest system and subsequent tests if it fails early.

 

I tried using the Try ... Finally keyword test steps but the Finally step isn't called when an error-level message is logged. Maybe we need to be able to raise exceptions from keyword tests instead of using Log Error Message, so the Finally step is still called.

Keyword Tests
Test Results

4 Comments

Sort By
  • hkim5's avatar
    hkim5
    Staff
    4 years ago

    it sounds like you are describing an event handler to me. It's my understanding that there is already a similar development underway, where the OnTestCaseStart/OnTestCaseEnd event handlers are getting revamped so that you can utilize them robustly as the type of setup/tear down scripts you are mentioning. this is due for 14.5 release (as far as I'm aware).

  • tphillips's avatar
    tphillips
    Frequent Contributor
    4 years ago

    That would be great. I did look into the event handlers but calling the same handler for every test would get out of hand fast.

    The current event handlers (onStartTest/onStopTest) are indeed not very useful because the aqTestCase.CurrentTestCase properly is set to null for event handlers, so you can't tell where they came from to decide what to clean up.

    If there is something like this already in development then that's great! Looking forward to it in an upcoming release!

  • hkim5's avatar
    hkim5
    Staff
    4 years ago

    I think the reason you are currently getting the null error for aqTestCase.CurrentTestCase is that you are running the test cases at the test level with the associated event handler. I think you've also noticed the behavior that the null error does not occur when you run it at the project level with the desired test cases. This happens because the event handler kicks in after the test case (at least for certain event handlers) which means that there are no test cases, hence no aqTestCase (hence null). 

    But in terms of "calling the same handler for every test", this will still stand, because event handlers are created precisely for that reason (so that the same thing occurs EVERY time you run the test). The updated event handlers (OnTestCaseStart/OnTestCaseEnd event) will have additional properties (namely 

    parameters:

    • Test case status (passed/warning/failed)

    • The first error object with properties:

      • Message

      • Additional message

      • Picture (if "screenshot on error" option is enabled) that we should be able to incorporate into our event handlers as needed. In terms of selective set up/teardown methods, I'd look into create short keyword tests or scripts to meet that need. I'd love to have more conversations with you to understand the your use case, and see if I can have more discussions with our product owner 🙂