GUI changes in regression test
Hi all, We have (and expanding) a test suite in our organization to do regression testing. I have a problem that every time there is a new build /release, the test crash since there are some GUI...
Forum Discussion
"And really, a test crashing because the program changed is a good thing, isn't it?"
Personally, I 100% disagree with this. Failing - yes. Crashing - no.
As soon as automated tests start crashing, that's when management (amongst others) start to question the validity of them. Such has been my experience anyway.
I'd much rather the test kept running and my logs gave me good information about what didn't work. If it's a large fundamental change, the logs may contain nothing but failures. But more often than not, it's not and only part of the suite fails. The rest will carry on and run as normal.
Error handling, recovery, error handling, recovery ..... it's like a mantra for building a new module! :)
And as Marsha says, it's handy if your dev people also let you know about potential breaking changes as early as possible. Some are better than others at this ....
Personally, I 100% disagree with this. Failing - yes. Crashing - no.
As soon as automated tests start crashing, that's when management (amongst others) start to question the validity of them. Such has been my experience anyway.
I'd much rather the test kept running and my logs gave me good information about what didn't work. If it's a large fundamental change, the logs may contain nothing but failures. But more often than not, it's not and only part of the suite fails. The rest will carry on and run as normal.
Error handling, recovery, error handling, recovery ..... it's like a mantra for building a new module! :)
And as Marsha says, it's handy if your dev people also let you know about potential breaking changes as early as possible. Some are better than others at this ....
