Zephyr - Test execution report

Hi,

There is a user story with 5 test cases and I executed 3 test cases out of 5. How can I see zephyr execution report with total 5 tests and 3 are executed with pass/fail status?

If I start executing a test and not completed the execution then it is showing as 'Not executed' in test report. But what about test cases which were not at all executed? From which report I number of test cases executed versus test cases which were not started execution at all?

Thanks in advance,

Avanthi

Zephyr Scale (general)

MisterB
2 years ago
Some things that might explain the difference:

You have 692 test cases but only 398 test cases have been added to test cycles. Your report is correctly showing all test cases that exist in all test cycles

Your report logic does not include all Test Cycles (if that's your criteria), or if you're using Test Plan as criteria, not all Test Cycles have been added into the Test Plan

Your report logic has other criteria that is excluding Test Cases, e.g. you're excluding Test Cases that have a Result of Not Executed

There could be more reasons for this but it's hard to tell without seeing more. If none of the above helps, can you please send a screenshot of the gadget criteria/logic screen.

MisterB
Champion Level 3
2 years ago
Hi,

The lifecycle of a test case execution status is: Not Executed > In Progress > Done. So in your scenario, if you started executing 1 of your 2 Not Executed tests but didn't complete it, you should see that the test execution status display as "In Progress". If you're viewing the status from within the Story then you should see the grey (Not Executed) status button change to Orange (In Progress).

Does that answer your question?
- Avanthideviredd
  New Contributor
  2 years ago
  Thanks for your reply. Whatever you mentioned is correct. But in Test execution reports, in progress test cases are showing as not executed and Not executed test cases are not showing at all. Is it a bug in Zephyr reports?
  - MisterB
    Champion Level 3
    2 years ago
    I see what you mean. I think the report is correct but it depends on your definition of what Remaining means. The report considers a test as Remaining if they are 'Not Executed' OR 'In Progress'. Whereas tests that are Completed are Fail, Pass, and Blocked (I think this a custom status I created).
    
    It might help to see the same charts with more information. Both charts below show 32 test cases. The Blue and Red statuses have 1 test case each, and it's easy to see that 7 test cases are 'Complete', and 25 are 'In Progress'.
MisterB
Champion Level 3
2 years ago
Some things that might explain the difference:

You have 692 test cases but only 398 test cases have been added to test cycles. Your report is correctly showing all test cases that exist in all test cycles

Your report logic does not include all Test Cycles (if that's your criteria), or if you're using Test Plan as criteria, not all Test Cycles have been added into the Test Plan

Your report logic has other criteria that is excluding Test Cases, e.g. you're excluding Test Cases that have a Result of Not Executed

There could be more reasons for this but it's hard to tell without seeing more. If none of the above helps, can you please send a screenshot of the gadget criteria/logic screen.
- Avanthideviredd
  New Contributor
  2 years ago
  Got it. Thank you once again for answering all my questions.
  
  Thanks,
  Avanthi