Ask a Question

Test Results Reporting

Super Contributor

Test Results Reporting

I remember you mentioned that you are working on enhancing the reporting capabilities of soapUI. I am not what enhancement you have planned but I wanted to suggest a few based on talks with some management folks to figure out what they think is lacking in soapUI.

Based on my talks, I think management is mostly interested in nice graphs as opposed to actual results of the test run. One of the main reason for using tools like Quality Center, MSTFS, is that they can create some nice bar charts, pie charts etc. Based on this I think it will be helpful to have the following in soapUI

1. In our team, engineers are working on separate projects and the reports generated by soapUI are good for their own consumption. However, a manager would typically want to run all projects after a build and find out how each of those project ran. some of the reports should be
  a. Overall  (union of Project A,B,C..), test executed, not run (maybe they are disabled), test passed, test failed.
  b. Test passed/failed/not ran in each of the projects.
  c.  Comparison of the results with a previous build. Is the number of test failures going down as the project is moving forward.
  d. Graphs, Graphs, Graphs

I think if this functionality can be made part of soapUI, I would certainly not even use Mercury Quality Center. I think most corporations wont even mind paying extra for such a functionality.

One other thing I found out was the ability to track bugs within testcases. So if the testcases have a 'Issues/Bugs/Defects' tab in addition to the description tab, I can enter a bugID against a failing testcase. When the bug is  fixed I can disable the entry. I would like to keep a history of what bugs have been filed against that testcase to make sure that bugs are not being reintroduced after some build.

Hopefully you like some of these idea. Thanks again for your great work.

Ali Raza
Super Contributor

Hi Ali,

once again thank you for your great input.. one of the "problems" we face when starting to improve on the reporting functionality is the overwhelming possibility of what we could report, this really helps us "narrow down" reporting content and helps us understand the business requirements. We hope to improve on reporting in soapUI Pro 2.0 to be release in the beginning of December, but as a Pro user you will hopefully see this showing up in the snapshots in November..

If you come up with any more requirements/ideas in this regard.. please don't hesitate to communicate them..

regarding bug/issue-tracking.. Are you using some software for this (MQC,JIRA,etc?) or handling them "manually".. ?


Super Contributor

1. One requirement I forgot to mention about the 'business requirements' tab is to have a column for external links. some testcases relate to a use case which typically written by business units and uploaded to some location (network location, sharepoint, wiki etc). It would be helpful to click on this link and be directed to where the usecase or some other relevant document is.

2. Regarding bug tracking, we were initially using bugzilla but a lack of cool graphs made us switch to Microsoft Team Foundation Server. Notice the emphasis on 'cool' graphs   We have not quite made the transition yet to TFS and have to figure out how to port soapUI testcases to TFS. I am currently playing around with their SDK to see how it works. So at the moment there is no link between a testcase and a bug.


Showing results for 
Search instead for 
Did you mean: