Forum Discussion

Hebla's avatar
Hebla
Occasional Contributor
11 years ago

Jenkins, Maven + SoapUI. Need some config help

Hi

I've been struggeling a bit setting up Jenkins, Maven and SoapUI.

First off, here are my POM.xml


<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>

<name>Jenkins test</name>
<artifactId>MavenTestId</artifactId>
<groupId>JenkinsTestGroup</groupId>
<version>1.0-SNAPSHOT</version>
<packaging>jar</packaging>
<description>Jenkins test</description>
<build>
<plugins>
<plugin>
<groupId>com.smartbear.soapui</groupId>
<artifactId>soapui-maven-plugin</artifactId>
<version>4.6.1</version>
<executions>
<execution>
<id>LightLogin</id>
<goals>
<goal>test</goal>
</goals>
<phase>test</phase>
</execution>
</executions>
<configuration>
<projectFile>src/test/soapui/mNILL_FI.xml</projectFile>
<outputFolder>${project.build.directory}/surefire-reports</outputFolder>
<junitReport>true</junitReport>
<printReport>false</printReport>
<properties>
<testSuite>testingFI</testSuite>
</properties>
</configuration>
</plugin>
</plugins>
</build>
<reporting>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-report-plugin</artifactId>
<version>2.17</version>
<configuration>
<outputDirectory>${project.build.directory}/reports</outputDirectory>
<reportsDirectories>
<reportsDirectories>${project.build.directory}/reports</reportsDirectories>
</reportsDirectories>
</configuration>
</plugin>
</plugins>
</reporting>
</project>


First off, I can't get it to run the testSuite I want. Here is my SoapUI project:



I have the same testsuite as you can see here copied and modified slightly to see the different names in the testingFI testsuite as well.

No matter what I do, it insists on running both the testsuites, and if I try specifying a testcase, it runs that everything anyways. Now it does stop after the first testsuite, since there are failures when running it with maven, see more details below.

-----
Problem Nr 2.

I have a groovy script library in soapUI. However, this does not work when running the tests with Maven. I get:

15:06:11,556 ERROR [SoapUI] An error occurred [No such property: SOSvalidation for class: Script1], see error log for details
groovy.lang.MissingPropertyException: No such property: SOSvalidation for class: Script1

How can I hook this up?

-----
Problem nr 3.

JUnit reporting. When I run this in Maven command line, I can see the test run one by one, and I can see the testreports in the surefire-reports folder. When I run this in Jenkins, Jenkins only reports 2 tests run. I have no way of seeing how the individual tests went. To me it looks like there are no surefire-reports. I tried setting the surefire folders to /reports, instead of surefire-reports, but this folder is still empty. This is what I see in Jenkins:



Is this all I am supposed to see? The different errors are test-steps that try to use the groovy library... I have no reports on the other test-steps, or the other testsuite, the one that my POM is actually trying to run.

Do I have to make a testCase for each test I want a report on, instead of using teststeps? We have 50 services, that needs to be tested for different countries, so being able to use a test test-step as a Unit-test would be quite a lot better than having to use a testCase for each of the services(making setup a lot easier in this case).

I hope someone with a bit more experience than me can help me, I have been using a bit more time trying to figure this out that I thought I would...
  • redfish4ktc2's avatar
    redfish4ktc2
    Super Contributor
    hi,
    For problem 1
    to run a single testsuite, change your configuration
          <properties>
    <testSuite>testingFI</testSuite>
    </properties>

    by
    <testSuite>testingFI</testSuite>


    Work as well for testcase. See also this post: viewtopic.php?f=1&t=14992

    For problem 2
    In Soapui, where do you store the groovy script library? You have to make it available to the maven plugin. Several options are available, so could you provide more info?

    For problem 3
    I need more info.
    What type of jenkins job do you use (maven, freestyle)?
    you said you have no soapui logs in the jenkins console? have you check soapui log files in the workspace of your jenkins job?

    Soapui Junit reports produce 1 entry per test case. Basically, in the junit reports, Soapui testsuite are mapped to a class, and Soapui testcases are mapped to a method. So by default, there is no place for test steps.
    I guess in your Soapui project, you have configure your testcase to not fail on teststep failures to be able to run all steps, and you would like to know which steps failed.

    As this is not supported out of the box by soapui, this has been a feature request for a long time, see viewtopic.php?f=2&t=24405 (and a lot of post in this forum)
    As i state in viewtopic.php?f=5&t=19820, I have started implemented such a feature but this is currently not finished; are you interesting in testing it? I could deploy a snapshot of my plugin with the in progress feature.
    Remember, you won't have the failure information for the failing test step directly, this will be displayed at the test case level.
    See this example of report (with my custom impl) generated by ant
  • Hebla's avatar
    Hebla
    Occasional Contributor
    Thank you for your reply! the solution to problem 1 worked perfectly.

    Problem 2
    I keep my groovy script in a library folder, that I specify the location of in SoapUI (In the soapUI pro tab -> Script library). Can I specify this in Maven somehow?

    Problem 3
    I use Maven 3 to run the tests. I figured that it would be the best way to do it.

    I guess in your Soapui project, you have configure your testcase to not fail on teststep failures to be able to run all steps, and you would like to know which steps failed.


    Exactly. I am hoping to make a nice overview over available services. Basically my tests only needs to see if the response contains an element that the service would respond with, if it is up and running as it should be.

    From what I can understand, the easiest way would be to move my teststeps into testcases instead.

    I am not sure I can try your plugin. The company I'm doing this project for, has very strict rules about everything, so I have to use their local repositories, and ordering new plugins/addons/updates etc takes weeks. I would love for you to tell me when it is released/is in a stable version though, and I will definitely use it then. It looks like it is exactly what I am looking for!
  • Hebla's avatar
    Hebla
    Occasional Contributor
    I have now tried to add:

        <soapuiProperties>
    <property>
    <name>soapui.scripting.library</name>
    <value>src/test/library</value>
    </property>
    </soapuiProperties>


    I have copied my script to this folder. I get:

    Setting soapui.scripting.library value src/test/library


    in the console logg. It seems like it still can't use this library thought. I still get:

    14:23:59,131 ERROR [SoapUI] An error occurred [No such property: SOSvalidation for class: Script1], see error log for details
    groovy.lang.MissingPropertyException: No such property: SOSvalidation for class: Script1
    at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.unwrap(ScriptBytecodeAdapter.java:50)
    at org.codehaus.groovy.runtime.callsite.PogoGetPropertySite.getProperty(PogoGetPropertySite.java:49)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callGroovyObjectGetProperty(AbstractCallSite.java:231)
    at Script1.run(Script1.groovy:15)
    at com.eviware.soapui.support.scripting.groovy.SoapUIGroovyScriptEngine.run(SoapUIGroovyScriptEngine.java:96)
    at com.eviware.soapui.impl.wsdl.teststeps.WsdlGroovyScriptTestStep.run(WsdlGroovyScriptTestStep.java:149)
    at com.eviware.soapui.impl.wsdl.support.AbstractTestCaseRunner.runTestStep(AbstractTestCaseRunner.java:239)
    at com.eviware.soapui.impl.wsdl.testcase.WsdlTestCaseRunner.runCurrentTestStep(WsdlTestCaseRunner.java:48)
    at com.eviware.soapui.impl.wsdl.support.AbstractTestCaseRunner.internalRun(AbstractTestCaseRunner.java:148)
    at com.eviware.soapui.impl.wsdl.support.AbstractTestCaseRunner.internalRun(AbstractTestCaseRunner.java:43)
    at com.eviware.soapui.impl.wsdl.support.AbstractTestRunner.run(AbstractTestRunner.java:135)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
    at java.util.concurrent.FutureTask.run(FutureTask.java:262)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)


    Any additional help would be great:)
  • redfish4ktc2's avatar
    redfish4ktc2
    Super Contributor
    for problem 2
    the configuration for the script library is right, maybe you can use an absolute path to be sure (${basedir}/src/test/library) because I have already seen some issues when setting relative path when configuring the maven plugin
    does the class in your groovy script has a package? if so, you have to add subdirectories to match the package path as you may have done in the scripts directory used by the Soapui GUI


    for problem 3
    I will release the plugin in a stable version soon (i guess this week). The plugin is available on maven central, so you should be able to use it.
  • Hebla's avatar
    Hebla
    Occasional Contributor
    Thank you soo much for your help! I managed to figure out the groovy script yesterday. I had to use the pro version of the plugin. I didn't know it was 2 different ones. But I have to use the pro version of SoapUI to run the Groovy scripts, so I guess it makes sence to use the pro plugin for that as well:)

    I will talk to my employer, and see if they can put your plugin into their repository. They block internet for all applications on all their computers/servers, so I can't use any repositories except their own. I'm looking forward to seeing the results:)
  • brainiac007's avatar
    brainiac007
    Occasional Contributor
    Hi redfish4ktc2
    First of all thanks very much for all your contribution to this community!

    We are using the open source version of soapUI inside a Jenkins/Maven setup and running thousands of TestCases multiple times a day. Now we start moving the test parameters into Excel sheets because this it's much easier to change for Business Analysts.

    Based on this we have one (looping) TestStep only in a TestCase with reads the data from the Excel sheet. The Log shows all TestSteps but in the JUnit Report we have the TestCases only. Welcome to Problem 3.....

    What's the Status of your implementation - Issue #42? Is there a snapshot available for Testing?
  • redfish4ktc2's avatar
    redfish4ktc2
    Super Contributor
    @brainiac007, thanks for your feedback
    The implementation is almost done (the screenshot I previously posted has been produced with the current implem), I have to remove some debug logs and then I will deploy a snapshot
    Perhaps, this week. Otherwise, you should build the plugin from sources.
  • redfish4ktc2's avatar
    redfish4ktc2
    Super Contributor
    hi,
    i have deployed a 4.6.4.2-SNAPSHOT version of the plugin
    Get access to the plugin: https://github.com/redfish4ktc/maven-so ... hot-access
    Configuring the new junit collector:
    * see https://github.com/redfish4ktc/maven-so ... it-reports
    * the documentation will be improved based on your feedbacks
    * the class name of the collector is currently com.eviware.soapui.report.StepInfoJUnitReportCollector. It will change in the final version

    You can also check examples here: https://github.com/redfish4ktc/maven-so ... it_reports
    • season123's avatar
      season123
      Visitor

      Hello,

      I am trying to run SoapUi test suite from Jenkins for the first time. Does anyone has complete step by step procedure? Also I need maven plugin info.

      Thanks