cancel
Showing results for 
Search instead for 
Did you mean: 

Jenkins does not recognize ReadyApi maven plug-in test failures as unstable build

SOLVED
Highlighted
Occasional Contributor

Jenkins does not recognize ReadyApi maven plug-in test failures as unstable build

We're using Ready Api 2.3.0 with the ready-api-maven-plugin 2.3.0 to run our tests, which works fine.

Although some tests fail, Jenkins does not recognice the build as unstable, as it should when tests are failing.

 

ReadyAPI 2.3.0 TestCaseRunner Summary
-----------------------------
Time Taken: 16172ms
Total TestSuites: 0
Total TestCases: 2 (2 failed)
Total TestSteps: 686
Total Request Assertions: 5
Total Failed Assertions: 0
Total Exported Results: 2

19:40:57,286 INFO  [SoapUIProTestCaseRunner] Created report at file:/C:/vhvEntw/Jenkins/workspace/Testautomatisierung/SOAP_UI_Vertrieb/Vertrieb_SoapUI_TAA_Kfz_S/poms/kfz/s/target/TestOutput/index.html
[INFO] ReadyAPI 2.3.0 shutting down
[JENKINS] Zeichne Testergebnisse auf
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 47.889 s
[INFO] Finished at: 2019-07-25T19:40:57+02:00
[INFO] Final Memory: 78M/1189M
[INFO] ------------------------------------------------------------------------

 Here's the config that writes the results to the default maven location:

 

 

<configuration>
<projectFile>
../../../services/kfz/auto/TarifrechnerWS_KFZTarifrechnerService_v0118
</projectFile>
<projectProperties>
<value>taa_endpoint=${#Project#taa_Endpoint_S}</value>
<value>Verarbeitungsebene=S</value>
</projectProperties>
<testCase>Regression Tarif 01.10.18</testCase>
<junitReport>true</junitReport>
<testFailIgnore>true</testFailIgnore>
<printReport>true</printReport>
<reports>${basedir}/target/surefire-reports</reports>
<outputFolder>${basedir}/target/TestOutput</outputFolder>
</configuration>

content:

 

jenkins.PNG

 

Does anyone have an idea why Jenkins does not consider this build as unstable?

This works when using the soapui-maven-plugin the same way...

 

Thanks

Carsten

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted
Contributor

Re: Jenkins does not recognize ReadyApi maven plug-in test failures as unstable build

You are ignoring test failures in your plugin config in this line:

 

<testFailIgnore>true</testFailIgnore>

 

Actually, that's not quite correct, according to the documentation, this property determines if a test will continue running if it encounters an error:

 

Continues the test run even if an assertion triggers.
Possible values: true, false.

 

After your test runs, you can use the JUnit plugin to parse the results, and that will pass/fail the build.

 

 

View solution in original post

5 REPLIES 5
Highlighted
Contributor

Re: Jenkins does not recognize ReadyApi maven plug-in test failures as unstable build

You are ignoring test failures in your plugin config in this line:

 

<testFailIgnore>true</testFailIgnore>

 

Actually, that's not quite correct, according to the documentation, this property determines if a test will continue running if it encounters an error:

 

Continues the test run even if an assertion triggers.
Possible values: true, false.

 

After your test runs, you can use the JUnit plugin to parse the results, and that will pass/fail the build.

 

 

View solution in original post

Highlighted
Moderator

Re: Jenkins does not recognize ReadyApi maven plug-in test failures as unstable build

Thank you, @JustinM89!

 

@chilber, was this information helpful?


Olga Terentieva
SmartBear Assistant Community Manager

Highlighted
Occasional Contributor

Re: Jenkins does not recognize ReadyApi maven plug-in test failures as unstable build

Thanks for the answer, JustinM89.

But this does not adress my problem. I don't want the build to fail if a test fails, I want the build to be marked as unstable. Please consider the Jenkins terminology here: https://wiki.jenkins.io/display/JENKINS/Terminology

Thanks

Carsten

 

Highlighted
Contributor

Re: Jenkins does not recognize ReadyApi maven plug-in test failures as unstable build

I should clarify, the JUnit plugin will mark the build unstable (not failed) if there are any test failures.

Highlighted
Moderator

Re: Jenkins does not recognize ReadyApi maven plug-in test failures as unstable build

Thanks for the clarification, @JustinM89.

 

@chilber, does it work for you in this case?


Olga Terentieva
SmartBear Assistant Community Manager

New Here?
Join us and watch the welcome video:
Announcements
Top Kudoed Authors