Test Results Reporting
Hello,
I remember you mentioned that you are working on enhancing the reporting capabilities of soapUI. I am not what enhancement you have planned but I wanted to suggest a few based on talks with some management folks to figure out what they think is lacking in soapUI.
Based on my talks, I think management is mostly interested in nice graphs as opposed to actual results of the test run. One of the main reason for using tools like Quality Center, MSTFS, is that they can create some nice bar charts, pie charts etc. Based on this I think it will be helpful to have the following in soapUI
1. In our team, engineers are working on separate projects and the reports generated by soapUI are good for their own consumption. However, a manager would typically want to run all projects after a build and find out how each of those project ran. some of the reports should be
a. Overall (union of Project A,B,C..), test executed, not run (maybe they are disabled), test passed, test failed.
b. Test passed/failed/not ran in each of the projects.
c. Comparison of the results with a previous build. Is the number of test failures going down as the project is moving forward.
d. Graphs, Graphs, Graphs
I think if this functionality can be made part of soapUI, I would certainly not even use Mercury Quality Center. I think most corporations wont even mind paying extra for such a functionality.
One other thing I found out was the ability to track bugs within testcases. So if the testcases have a 'Issues/Bugs/Defects' tab in addition to the description tab, I can enter a bugID against a failing testcase. When the bug is fixed I can disable the entry. I would like to keep a history of what bugs have been filed against that testcase to make sure that bugs are not being reintroduced after some build.
Hopefully you like some of these idea. Thanks again for your great work.
Ali Raza
I remember you mentioned that you are working on enhancing the reporting capabilities of soapUI. I am not what enhancement you have planned but I wanted to suggest a few based on talks with some management folks to figure out what they think is lacking in soapUI.
Based on my talks, I think management is mostly interested in nice graphs as opposed to actual results of the test run. One of the main reason for using tools like Quality Center, MSTFS, is that they can create some nice bar charts, pie charts etc. Based on this I think it will be helpful to have the following in soapUI
1. In our team, engineers are working on separate projects and the reports generated by soapUI are good for their own consumption. However, a manager would typically want to run all projects after a build and find out how each of those project ran. some of the reports should be
a. Overall (union of Project A,B,C..), test executed, not run (maybe they are disabled), test passed, test failed.
b. Test passed/failed/not ran in each of the projects.
c. Comparison of the results with a previous build. Is the number of test failures going down as the project is moving forward.
d. Graphs, Graphs, Graphs
I think if this functionality can be made part of soapUI, I would certainly not even use Mercury Quality Center. I think most corporations wont even mind paying extra for such a functionality.
One other thing I found out was the ability to track bugs within testcases. So if the testcases have a 'Issues/Bugs/Defects' tab in addition to the description tab, I can enter a bugID against a failing testcase. When the bug is fixed I can disable the entry. I would like to keep a history of what bugs have been filed against that testcase to make sure that bugs are not being reintroduced after some build.
Hopefully you like some of these idea. Thanks again for your great work.
Ali Raza
We have not quite made the transition yet to TFS and have to figure out how to port soapUI testcases to TFS. I am currently playing around with their SDK to see how it works. So at the moment there is no link between a testcase and a bug.