Ask a Question

Jenkins does not recognize ReadyApi maven plug-in test failures as unstable build

SOLVED
chilber
Occasional Contributor

Jenkins does not recognize ReadyApi maven plug-in test failures as unstable build

We're using Ready Api 2.3.0 with the ready-api-maven-plugin 2.3.0 to run our tests, which works fine.

Although some tests fail, Jenkins does not recognice the build as unstable, as it should when tests are failing.

 

ReadyAPI 2.3.0 TestCaseRunner Summary
-----------------------------
Time Taken: 16172ms
Total TestSuites: 0
Total TestCases: 2 (2 failed)
Total TestSteps: 686
Total Request Assertions: 5
Total Failed Assertions: 0
Total Exported Results: 2

19:40:57,286 INFO  [SoapUIProTestCaseRunner] Created report at file:/C:/vhvEntw/Jenkins/workspace/Testautomatisierung/SOAP_UI_Vertrieb/Vertrieb_SoapUI_TAA_Kfz_S/poms/kfz/s/target/TestOutput/index.html
[INFO] ReadyAPI 2.3.0 shutting down
[JENKINS] Zeichne Testergebnisse auf
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 47.889 s
[INFO] Finished at: 2019-07-25T19:40:57+02:00
[INFO] Final Memory: 78M/1189M
[INFO] ------------------------------------------------------------------------

 Here's the config that writes the results to the default maven location:

 

 

<configuration>
<projectFile>
../../../services/kfz/auto/TarifrechnerWS_KFZTarifrechnerService_v0118
</projectFile>
<projectProperties>
<value>taa_endpoint=${#Project#taa_Endpoint_S}</value>
<value>Verarbeitungsebene=S</value>
</projectProperties>
<testCase>Regression Tarif 01.10.18</testCase>
<junitReport>true</junitReport>
<testFailIgnore>true</testFailIgnore>
<printReport>true</printReport>
<reports>${basedir}/target/surefire-reports</reports>
<outputFolder>${basedir}/target/TestOutput</outputFolder>
</configuration>

content:

 

jenkins.PNG

 

Does anyone have an idea why Jenkins does not consider this build as unstable?

This works when using the soapui-maven-plugin the same way...

 

Thanks

Carsten

5 REPLIES 5
JustinM89
Contributor

You are ignoring test failures in your plugin config in this line:

 

<testFailIgnore>true</testFailIgnore>

 

Actually, that's not quite correct, according to the documentation, this property determines if a test will continue running if it encounters an error:

 

Continues the test run even if an assertion triggers.
Possible values: true, false.

 

After your test runs, you can use the JUnit plugin to parse the results, and that will pass/fail the build.

 

 

Olga_T
SmartBear Alumni (Retired)

Thank you, @JustinM89!

 

@chilber, was this information helpful?


Olga Terentieva
SmartBear Assistant Community Manager

chilber
Occasional Contributor

Thanks for the answer, JustinM89.

But this does not adress my problem. I don't want the build to fail if a test fails, I want the build to be marked as unstable. Please consider the Jenkins terminology here: https://wiki.jenkins.io/display/JENKINS/Terminology

Thanks

Carsten

 

I should clarify, the JUnit plugin will mark the build unstable (not failed) if there are any test failures.

Olga_T
SmartBear Alumni (Retired)

Thanks for the clarification, @JustinM89.

 

@chilber, does it work for you in this case?


Olga Terentieva
SmartBear Assistant Community Manager

cancel
Showing results for 
Search instead for 
Did you mean: