Negative testing
Hello, it's me again... in our test suite we have to turn off some tests, because the bugs are fixed yet. However, it is desirable to keep all tests active, to ensure that all test...
Forum Discussion
We do not want to stop TestComplete. All tests should run through.
We evaluate the results of tests on the end from the logfiles:
<Prp Name="status" type="I" value="2"/>
"Value=2" indicates that an error has occurred.
The idea is rather in the direction of a new column for tc test-suite that says:
if error occurs ignore, else if NOT an error occurs then report as an error.
the alternative way: one have to negate the result in the code.
e.g.
procedure myBuggyFunction;
begin
// ... o my god an error occured!
aqObject.CompareProperty(DiffMem, cmpLessOrEqual, MemDiff_SoftPlayer, true, lmError);
end;
this would report me an Error in the log-file.
but we know, that dies test fails because the bug is not debugged.
i have to reform above code into this code to transfer my idea into reality:
procedure myBuggyFunction;
begin
// ... o my god an error occured!
ERRORLEVEL := lmError;
if IgnoreTestResult then
ERRORLEVEL := lmWarning;
aqObject.CompareProperty(DiffMem, cmpLessOrEqual, MemDiff_SoftPlayer, true, ERRORLEVEL);
end;
... and this additional code i have to write in any test!
