Forum Discussion

escap89's avatar
escap89
Contributor
9 years ago

How to auto run some script if some test in our test plan fail.

Hello!

 

I have for example 5 tests like:

TEST A: Open Application

TEST B: Add some user

TEST C: Edit some user

TEST D: Remove some user

TEST E: Close Application

 

Lets say that i config all tests like:

Stop on error: TestItem

 

Every our test expect Open and Start application expecting first window we see after we open our application and we always leave it as it on our last step in every test.

 

If "TEST A" for example stop on error then every next test will also fail because they expecting some other window at start.

 

Because of that i want to make some script that will:

Close windows process (our desktop application) and then run "TEST A".

 

I just dont know how to connect that script with "Stop on error: TestItem"

So then it would be like:

If "TEST A" for example stop on error we run our new script "close process and run test A" and

then next tests in our test plan will be executed.

 

How u guys handle with situations like this?

 

Any advice will be useful!:)

 

//Sebastian

 

 

  • Bobik's avatar
    Bobik
    9 years ago

    I see 2 solutions:
    1. Add this logic into script. And run your tests from script instead of test items.
    2. Use project or project suite variables as flags. Next test will just stop if flag was set. In this case logic should be added into existing tests. You can set flags inside OnLogError event handler.

  • cunderw's avatar
    cunderw
    9 years ago

    My solution to this was to use the on test start and on test stop event, it checks that the AUT is on the starting screen. All tests are written to start and stop on this screen. If it any test is not on the start screen, the event handlers restart the application.

     

    You could also use the on log error event handler. 

  • NisHera's avatar
    NisHera
    9 years ago

    this is a usual dilemma faced by testers who start automate testing.
    The way you should design will depend on what you need to test.

     

    For eg. in your example if TEST A failed (Opening application) there are no way of running TEST -B
    how could you run other tests if application failed to open?..will it be pass second time if you do same?

     

    But in second case if you pass TEST A and failed TEST B (Add user) you can still run TEST C with some existing user.
    I would rather design TEST C independent of TEST B (newly add user not used in TEST C)
    So with the TEST D

     

    My advice is make it simple. Design each test as much as independent.
    So even TEST B fails your TEST C will run.
    Also design a set of smoke tests (link) which runs b'for actual test run
    In your eg TEST A and TEST E can be part of smoke tests.

  • cunderw's avatar
    cunderw
    Community Hero
    9 years ago

    My solution to this was to use the on test start and on test stop event, it checks that the AUT is on the starting screen. All tests are written to start and stop on this screen. If it any test is not on the start screen, the event handlers restart the application.

     

    You could also use the on log error event handler. 

  • NisHera's avatar
    NisHera
    Valued Contributor
    9 years ago

    this is a usual dilemma faced by testers who start automate testing.
    The way you should design will depend on what you need to test.

     

    For eg. in your example if TEST A failed (Opening application) there are no way of running TEST -B
    how could you run other tests if application failed to open?..will it be pass second time if you do same?

     

    But in second case if you pass TEST A and failed TEST B (Add user) you can still run TEST C with some existing user.
    I would rather design TEST C independent of TEST B (newly add user not used in TEST C)
    So with the TEST D

     

    My advice is make it simple. Design each test as much as independent.
    So even TEST B fails your TEST C will run.
    Also design a set of smoke tests (link) which runs b'for actual test run
    In your eg TEST A and TEST E can be part of smoke tests.

  • Bobik's avatar
    Bobik
    Frequent Contributor
    9 years ago

    I see 2 solutions:
    1. Add this logic into script. And run your tests from script instead of test items.
    2. Use project or project suite variables as flags. Next test will just stop if flag was set. In this case logic should be added into existing tests. You can set flags inside OnLogError event handler.

  • escap89's avatar
    escap89
    Contributor
    9 years ago

    Bobik

    I was thinking about add some logic into all my test scripts but i wanted avoid it.

    cunderw

    I think i will use it too. Just made test and it works exacly as i wants before every test i will check if app is on main window.

    NisHera
    I am creating tests independent as much as i can but in start/close application i wanted to avoid restarting app after every test.For example i have print document test and if i would want to make this on doc i created in previous step it would be rlly hard to handle all cases that can happen :) (if there is actual date printed document for example).

    So after reading all ur advices i decided to try use Test Enigine Events - OnStartTest.

    Like:
    abstract code:

    import os
import OpenAppTest
def GeneralEvents_OnStartTest(Sender):
If (mainwindow=true):
pass
else:
filepath="...\\checkprocess.bat"
fh = os.popen(filepath "application.exe")
output=fh.read()
if "SUCCESS" in output:
Log.Message(output)
Log.Message("Killed process application.exe")
else:
Log.Warning(output)
Log.Error("Didn't found process application.exe")
fh.close()
OpenAppTest.main()

    Thanks for all advices.
    :)

  • escap89's avatar
    escap89
    Contributor
    9 years ago

    Well, because i dont want to make exception for test A and test E i wont run there test separated as i do now.

    BeforeTest Even will open app if its needed.