Forum Discussion

mnwill117's avatar
mnwill117
Occasional Contributor
1 hour ago

Managing Failures - New vs. Expected/Known Issues

Our test suites run on a schedule in multiple environments, using test runner to run tests using tags to run test cases across multiple test suites. 

We're struggling to monitor all our test executions, and distinguish between new failures, that may be outages, new bugs, or issue with our tests, and known issues that are bugs that haven't been fixed yet, but we know we've reported and are tracking. 

How do you manage distinguishing ReadyAPI failures between ones that need to be looked at and triaged, and ones that are expected failures due to known issues?

I wish there was an additional status for 'warning' in addition to pass/fail. 

What other ideas have your teams used to distinguish known issues?

Some ideas, mostly bad - 

  • tag test cases with known issues (not granular enough as we're seeing issues at a test step level)
  • use a script to change a known issue's test step status to 'unknown' instead of fail
No RepliesBe the first to reply