Forum Discussion

SlickRick's avatar
SlickRick
Contributor
10 months ago

Flagging Known Bugs

So i wrote some UI tests for some feature which have known bugs. Obviously those tests are currently failing because indeed the feature is broken. However in my azure pipeline i would those tests...
Desktop Testing
  • MW_Didata's avatar
    MW_Didata
    10 months ago

    I like to post a warning on top and disable the actions inside the Keywordtest, this way you can see what tests are disabld by the warning, and then just remove the warning an re-enable when it gets fixed: 

    If you use a ticket system you could post the ticket number in the warning.

SlickRick's avatar
SlickRick
Contributor

But the purpose is not to exclude them from the test run, but to show them differently.

A bit like TDD, where the product is not considered complete until all tests are fully passing.

The idea here to differentiate between a new failing test (not known) vs a test that fails due to a bug which was already discovered but waiting to be fixed.

Anyways, it seems like I'm the only one seeing value in this :)

rraghvani's avatar
rraghvani
Champion Level 3
10 months ago

I have a bunch of tests which are dependant on one another, and they need to be executed in order, using the execution plan. If a test fails, it will usually cause the other tests to fail, even though I have On Error set to Continue Running. However, if the remaining tests pass, I won't accept that as a pass. Until the issue has been fixed and the test that failed, actually passes. Certain situations may give a false positive or negative result. 

It's best to keep the automation as simple as possible.