Test Coverage Run Details by Test Set Report
Hi,
I have a regression setup with 30 'sub-folders' that are my test sets with each containing additional sub-folders for various specific areas to test and those of course contain the actual tests to run. If I go to Test Management > Test Sets > Test Set parent folder I will see a listing of all test sets on the right with a column listing how many tests are in each test set. When I total this number up it is accurate.
When I run the Test Coverage Run Details by Test Set report it breaks down the tests into different states such as passed, in progress, failed, and so on but the numbers for each category don't make sense as they add up to be a lot more than the total number of tests in the regression...either I don't understand how to read this report or perhaps something is off.
Does anyone else notice this also?
Hi -- In my mind this report is essentially a historical view of test execution by test set for your filtered release. Meaning it will show all executions and all statuses for all test sets in a given release. This is why the numbers in the report are greater than you expect. Any report (crystal or ad-hoc) that has the word “Run” in it will give you a historical view.
Both of the crystal reports (Test Coverage Run Details by Test Set and Test Coverage Run Summary by Test Set) have limited filtering options and only allow you to filter down to the just a release level. If you could filter down to release AND last execution status this report it would make this report more meaningful.
If I want a current execution status for my test sets in a given release I run an ad-hoc Test Sets Summary report and group it appropriately.
Charlie
