IT is up to you, as the developer of the automation, to make that determinination in how you've implemented your code. You should handle errors in code to determine "Is this a test case failure or is this a code exception?"
IMO, technically, a code exception IS a test case failure... the test case failed to complete properly with a "Green" status. This could be due to an unhandled situation (new dialog, redesigned UI, etc) or an actual code error (type mismatch, etc). Something happened that was counter to the expected results.
If a human being were following a documented manual test and the documentation didn't match up to the application, that would ALSO be a failed test case...in that situation, the "code" is the documented test case that was incorrect... sure, it wasn't an actual PROBLEM with the application under test, but it WAS a failed test case indicating a required mitigation.
Related Content
- 3 years ago
- 8 years ago
- 11 years ago