Forum Discussion

djangofan's avatar
djangofan
Contributor
12 years ago

How do I begin to implement a custom JUnitReportCollector?

Maybe biting off more than I can chew, but I am looking for advice on how to implement the new feature in maven-soapui-extension-plugin for "configuring the JunitReportCollector to be able to modify xml junit files generation" but I am having trouble figuring out where to start.

I basically want a ReportCollector that is capable of printing out the test steps (in the HTML report) as it runs through each test case within a suite. I am not sure this is even possible. I can do this, I just need a basic idea of where to start. I have read all the related threads and help docs on this matter and everything is very vague.

Here is my project template, which is the point I want to start from, in order to implement this:
https://github.com/djangofan/soapui-test

I am guessing this might involve building sub-nodes into the report.xml "testcase" node, like so?


....
    <testcase name="RequestTime TestCase" time="0.292">
        <teststep name="Step1" time="0.122"/>
        <teststep name="Step2">
            <failure type="Assertion" message="Test failure.">That statement is not true!</failure>
        </teststep>
        <teststep name="Step3">
            <failure type="Exception">java.lang.NullPointerException</failure>
        </teststep>
    </testcase>
  </testsuite>
</testsuites>


If this is not possible, then the workaround, of course, is to put each of my test steps into it's own test case. This would be perfectly acceptable if I could attach a link to specific console output to each test case.
  • djangofan's avatar
    djangofan
    Contributor
    12 years ago
    Yes, I saw that post but nobody seems to be using the maven-soapui-plugin like I am, and since those suggestions are out of context, I am at a loss as to how to do this, for example, by using a Groovy script step or a Groovy class defined outside of a SoapUI project.

    My workaround is simple. I think I was incorrectly adding “tests” in SoapUI as “test steps within a case” instead of adding them
    as separate test cases , each containing one step. This was causing my report to have no detail about each individual test case.

    So, my opinion is that my original question (above) is something that nobody has yet done, and may not be possible.
  • nmrao's avatar
    nmrao
    Champion Level 3
    12 years ago
    I am novoice at maven.

    Using ant+junit reports, they are no so detailed though. Am also looking for some resources to improve report by customizing.

    Are you able to generate any kind of reports with what you have so far?
  • djangofan's avatar
    djangofan
    Contributor
    12 years ago
    Yes, I get a report generated.

    The report generated by my project (see above URL) gives me the number of test cases but only shows test case names if the test fails. The report doesn't show me successful test case names, which I wish it would do. As you can see from my configuration, I have set "exportAll" as a Maven option to the plugin but it seems that SoapUI is ignoring that option.
  • redfish4ktc2's avatar
    redfish4ktc2
    Super Contributor
    12 years ago
    Hi guys,
    nice to see someone ready to contribute to https://github.com/redfish4ktc/maven-so ... ssues/42!!! :-)

    Here are some information I have collected
    exportAll: this option is not related to junit report. Soapui generates txt files that contains the details of failing steps (request, response, assertions result). When set to true, soapui generates txt files also for successful steps.
    This is implemented directly in the test runner and this behaviour is not specific to Maven. See the source code https://github.com/SmartBear/soapui/blo ... .java#L683

    About the JunitReportCollector
    I discovered yesterday an hidden feature: it is possible to get the detail of a failure in the junit report by setting the "Complete Error Logs" property to true.
    See
    * viewtopic.php?f=2&t=2313
    * viewtopic.php?f=4&t=2448
    This is a pro feature in the gui, but you can create an empty soapui settings file, add the property and make it use by the maven plugin. This should work even with the open source version.
    The content of the TestStepResult is written directly as this is done when generating the txt files
    I did not have time to test this to see what it produces, maybe later this summer

    The source code of the junit collector is here: https://github.com/SmartBear/soapui/blo ... .java#L139
    It explains that you only get failures details in the report because all the implementation does not consider any success details
    * the afterStep method perform some works only on failures. There is no field that stores successes step or tescases
    * the afterRun method shows that the report class currently used stores test details only for failures

    This is also what it is explain here: http://blog.infostretch.com/customizing-soapui-reports. They did not give all the implementation but if I had time to implement this feature right now, I would look deeply into this post: it show how to display all steps, expecially successful step and all assertions (this seems to be inspired of the way the testrunner deals with result in its afterStep method).
    The details of the step are directly sent to the report related to the current testsuite through sysout and syserr methods


    Hope this help

    djangofan can you share your implementation on github/gist for letting me have a look at it? Thanks
  • nmrao's avatar
    nmrao
    Champion Level 3
    12 years ago
    @redfish4ktc
    Thank you for your time to write in detailed. Even, earlier I saw your maven plugin on github through different posts on this report generation topics, which is nice. As I mentioned in other post where you have provided multiple links, I like that blog too and that was not presented in detailed. I am curious that you plan to implement it, looking forward to it. Not a regurlar java programmer, but some hands on java, sopaui & groovy.
  • djangofan's avatar
    djangofan
    Contributor
    12 years ago
    redfish4ktc2 - Thanks for the answer and the links. My GitHub url is shared in the initial post on this thread. In that project I am using your extension plugin. I discovered a "workaround" for my problem by moving test steps into their own test cases (each test case having one step). I still would like to figure out a way to log individual steps but I am not feeling very confident that I will be successful. Feel free to fork my project and give me a pull request.
  • geekay's avatar
    geekay
    New Contributor
    10 years ago
    Hi,
    I am a die hard fan of soapUI and new to this forum.

    I am interested to implement this customized junit reporting feature in soapUI opensource. I have tried to extend both JunitReportCollector and JUnitSecurityReportCollector class and overriden afterRun/afterStep but it did not work. What i need are-

    - want to display the testcase description instead of testcase name in the junit report xml
    - so what jar should i first take (is it soapui-5.0.0.jar in the SOAPUI bin directory?)
    - what class i shud extend ? JUnitSecurityReportCollector or JunitReportCollector ?
    - what field i need to change so that testcase description will be displayed
    - once changes done, am i required to generate same soapui-5.0.0.jar and place in same path?

    Kindly advise on these. VALUABLE RESPONSES ARE MUCH APPRECIATED!!!