Hi,
I'm not sure if it's what you're looking for but there is a gadget called "Test execution results by type" which seems to distinguish between manual and automated execution. My test cases are all manual so I'm unable to validate but it looks like it might be what you need. You can then access your custom fields as criteria using the "Filter by" "Custom Query" option.
Hope it helps.
Andy
Hello,
Thanks for your answer , but I think it doesn't solve my problem. 😞
Let me clarify my problem again, but from a different point of view:
Let's suppose that there is a SUT (Sw Under Test), which covered by 1000 test cases stored in Zephyr Scale.
Test automation started late, there are some test cases marked with "need to be automated" custom field - let's say the number of these is 200 test cases.
And let's suppose that 50 test cases already automated.
I am searching for a solution how I am able to visualize test automation's progress in Zephyr Scale.
So currently I would need a diagram \ gadget which contains 200 test cases, with the following legend:
- 50 test cases marked as done \ automated
- 150 test cases marked as not yet automated
So I am curious about how many test cases covered by automation already and how many is left.
Maybe the usage of a custom field (for "need to be automated") is a dead end (for visualization), and this is why I asked in the 1st comment "What is the best practice for visualization of automated \ not yet automated test cases?"
Kr,
Andras
Hi Andras,
I understand what you're trying to achieve.
I'm wondering if the Test Case Status field would help, along with the report Test Library > Test cases by status?
In my instance I have 3 test case statuses: Draft, Deprecated, Approved. I'm thinking you could add or alter these statuses to include your two scenarios, e.g. Automated, and Not Automated. To do that, click on the Zephyr Scale cog (not the Jira cog) and you'll see a Statuses section with one for Test Cases. You can add them there.
You would then change the status of the test cases accordingly and the gadget/report mentioned above should work, showing you a donut chart of the split between the different statuses.
Let me know how it goes!
Cheers, Andy
Hello,
Thanks for the tip! Yeah, we also considered this. It can work, but we wanted a better solution. Let me clarify why:
Current values of Test Case Statuses field are:
If we extend this with the automation statuses, like:
Than we will have too much statuses (9) without proper workflow model (I mean -if I am right- there is no way to create a real workflow like in Jira), so the people can easily get confused, which is unlucky.
I don't want to be insatiable, but any another idea? 🙂 (If no, I can accept this answer as a solution.)
Hi,
I can understand why you wouldn't want to do that 😉. You could look at the other fields in the Test Case screen and see what you could potentially use as criteria for reporting. I don't know what you're using but the fields listed below would all work as they are are accessible as criteria in reports. What you're looking for in the reports is one that has the Custom Query option and the ability to add criteria for Test Cases - this is where you would be able to add criteria for the fields suggested below:
- Components (intended for a different purpose but it will work and can be defined in the Project Settings)
- Priorities (if you're not using this works very nicely in some reports)
- Labels (this could be your best option because it's setup nicely to have multiple label values that I don't think will interfere with other purposes)
The way I see it working is that you would create 3 reports or gadgets, 1 each for Automation Needed, Automation In Progress, and Automated. You could then show these alongside in a dashboard and that would give you total numbers for scenario.
Some examples:
Test Execution Scorecard by Component - even though it's purpose is to show execution progress, it can still be used to separate between component values
Test Cases Summary - has a donut dashboard element that shows the split of test cases by Priority
Test Execution Results (Progress) - is a donut chart that is also available as a dashboard gadget. It's intended to show execution status but you can use criteria such as Label, Priority, or Component to return a total number of test cases, ignoring the test execution status results. Not that great though if you have multiple execution statuses, although you could add extra criteria to show only those test cases that haven't been executed if that's an option.
Test Execution Results (Overall) - similar to the above report; also available as a dashboard gadget.
Test Cases by Status - this will also work in the same way as the above. Also available as a dashboard gadget
Hope one of those helps.
