Forum Discussion

sudram's avatar
sudram
New Contributor
12 years ago

Enhancing the reports in Jenkins

Currently, I have a Jenkins job configured to use an ant build task to execute the testrunner.bat to run my SOAP UI tests suite and it all works. The test report information in Jenkins are pretty shallow though. I configured the JUnitReport ant task and the test information though slightly better is still far from what I'd like. I figured I may have two options to customize the reporting:

1. to override the JUnitReportCollector to customize the reporting (if it is possible to specify my own over-ridden JUnitReportCollector as a parameter to testrunner.bat?)
2. to use the listeners capability

I'm using SOAP UI (not Pro). Here are the things I'm hoping to achieve:
1. Each test case should also list all test steps
2. Each test step should list all it's assertions regardless of success or failure
3. Each test step that has a soap request/response should be hyper-linked to view this

Would appreciate pointers on whether overriding the JUnitReportCollector or using the Listeners capability is the way to go to achieve my needs. And any pointers on how to go about implementing the list of items I've listed above.
  • sudram's avatar
    sudram
    New Contributor
    Thanks. I'd come across that blog post before. So sounds like overriding the JUnitReportCollector may be the way to go. If that is the case, is there a way to provide my custom JUnitReportCollector class as an argument to testrunner.bat?
  • sudram's avatar
    sudram
    New Contributor
    Thanks for the tip. I will look into the TestRunner code to see if I can use that somehow.

    Meanwhile, I came across the https://github.com/SmartBear/soapui/blo ... ector.java and the createNew method in there. Looks like this guy checks for a soapui.junit.reportCollector system property and uses that if available. I haven't had any luck getting it to work, but was wondering if anyone has used this to pass in their own custom JUnitReportCollector.

    As far as using a listener, I couldn't figure out how to manipulate the JUnitReport from a listener. Might research this some more.
  • brainiac007's avatar
    brainiac007
    Occasional Contributor
    Hi sudram
    I'm also very interested in this but until now I din't find a working solution.
  • i have good experience in java, i.e. when you import in java you have a corresponding jar file to it. with Groovy i still dont understand/know what class to import.
  • redfish4ktc2's avatar
    redfish4ktc2
    Super Contributor
    Hi, as my account has been deleted, here is a summary of my deleted post
    * we could do what is explain in this post: viewtopic.php?f=2&t=17209
    * the createNew method factory of the JunitReportCollector (https://github.com/SmartBear/soapui/blo ... .java#L271) is not used
    * before soapui 4.0.0, there was a way to extend the testcase runner by overriding the createJUnitReportCollector method (see https://github.com/SmartBear/soapui/com ... 5c888b16a9) but this has been removed when the security-test feature has been introduced.
    * so to be able to use a custom JunitReportCollector, we should 1st override the or restore a way to configure the collector "a la" createNew method (notice that we should return an instance of JUnitSecurityReportCollector)
  • redfish4ktc2's avatar
    redfish4ktc2
    Super Contributor
    Hi,
    for maven users
    I've implemented a way to configure the JUnitReportCollector class which is used by the maven plugin. This will be available in the next 4.5.1.4 release of maven-soapui-extension-plugin (see https://github.com/redfish4ktc/maven-so ... it-reports)

    for others
    The implementation is available on github
    custom junit report collector selection: https://github.com/redfish4ktc/maven-so ... mpl/report
    custom SoapuiTestCaseRunner: https://github.com/redfish4ktc/maven-so ... r.java#L52

    enjoy!
  • Hi,

    I have a similar requirement as mentioned in the 1st post. I have extended the JUnitSecurityReportCollector, instantiated it and added extra info to the failures/success. The problem i am facing is with this requirement/customization :
    -" Each test step that has a soap request/response should be hyper-linked to view this "

    Here is snippet from my custom ReportCollector which 'tries' to achieve this.(from this post: viewtopic.php?f=2&t=20827 )

    if (result.getStatus() == TestStepStatus.FAILED) {

    buf.append("\t\tRequest/Response File path: <a href=\""
    + pathToFile + "\"> ShowFile </a>");
    }

    Now here is where the problem lies. The generated xml report file - escapes the '<' and replaces it &lt;. (Rightly so i guess).
    Snippet from generated xml report:

    <failure...>... Request/Response File path: &lt;a href="blah blah path to file"> ShowFile &lt;/a> ... </failure>

    When i build the this into html report using junitreport ant task - the '>' gets escaped too in resulting html doc. The resulting html page look like attached image

    I have tried the below 2 options :
    - adding a CDATA section so < isnt escaped (Same Output as above)
    -removed the xml &lt; and replaced it with '<' for the particalar a href tag. (Resulting html only displaying "Show File" as a plain text. Content of <a href is completely ignored - probably as it doesn't recognize the <a tag)

    I have also gone through all the numerous forum posts and blogs regarding customizing the report. (It's thanks to those that i got this far anyway!) All of them seem to mention that they want a link to screenshots or request/response - but none unfortunately explain this part/problem. Each one is centered around providing a custom JunitReportCollector to add relevant extra info.

    The only probable solutions i see
    - manually overriding the junit-frames.xsl (or junit-noframes.xsl) - to somehow figure out that whenever &lt;a is encountered - create an html link. (i am not sure if this will solve the problem either)
    - parse the resulting html and reverse the replacement of &lt;,&gt; for the relevant portion.

    Has anyone else encountered this (and hopefully solved it) ? Am i missing something ? I am not too great with xslt - so am trying to figure out other ways.
    Any help is MUCH appreciated!!
  • redfish4ktc2's avatar
    redfish4ktc2
    Super Contributor
    hi, did you find any valuable information to solve your problem?
    I am also working on enhancing reports (see my contributions in various posts in these forums)
  • redfish4ktc2's avatar
    redfish4ktc2
    Super Contributor
    purplehaze007 I have maybe a workaround for you
    what i have seen in your example is that you want to add a link to the request/response txt file generated by soapuI
    As a workaround, you can tell soapui to add to failing tests this information by setting the "Complete Error Logs". See my post on this subject: viewtopic.php?f=5&t=19820#p45262
    I have testing it: currently, I have been able to make it works only by adding the setting in the settings.xml soapui settings file