Jenkins does not recognize ReadyApi maven plug-in test failures as unstable build
We're using Ready Api 2.3.0 with the ready-api-maven-plugin 2.3.0 to run our tests, which works fine.
Although some tests fail, Jenkins does not recognice the build as unstable, as it should when tests are failing.
ReadyAPI 2.3.0 TestCaseRunner Summary ----------------------------- Time Taken: 16172ms Total TestSuites: 0 Total TestCases: 2 (2 failed) Total TestSteps: 686 Total Request Assertions: 5 Total Failed Assertions: 0 Total Exported Results: 2 19:40:57,286 INFO [SoapUIProTestCaseRunner] Created report at file:/C:/vhvEntw/Jenkins/workspace/Testautomatisierung/SOAP_UI_Vertrieb/Vertrieb_SoapUI_TAA_Kfz_S/poms/kfz/s/target/TestOutput/index.html [INFO] ReadyAPI 2.3.0 shutting down [JENKINS] Zeichne Testergebnisse auf [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 47.889 s [INFO] Finished at: 2019-07-25T19:40:57+02:00 [INFO] Final Memory: 78M/1189M [INFO] ------------------------------------------------------------------------
Here's the config that writes the results to the default maven location:
<configuration>
<projectFile>
../../../services/kfz/auto/TarifrechnerWS_KFZTarifrechnerService_v0118
</projectFile>
<projectProperties>
<value>taa_endpoint=${#Project#taa_Endpoint_S}</value>
<value>Verarbeitungsebene=S</value>
</projectProperties>
<testCase>Regression Tarif 01.10.18</testCase>
<junitReport>true</junitReport>
<testFailIgnore>true</testFailIgnore>
<printReport>true</printReport>
<reports>${basedir}/target/surefire-reports</reports>
<outputFolder>${basedir}/target/TestOutput</outputFolder>
</configuration>
content:
Does anyone have an idea why Jenkins does not consider this build as unstable?
This works when using the soapui-maven-plugin the same way...
Thanks
Carsten
You are ignoring test failures in your plugin config in this line:
<testFailIgnore>true</testFailIgnore>
Actually, that's not quite correct, according to the documentation, this property determines if a test will continue running if it encounters an error:
Continues the test run even if an assertion triggers. Possible values: true, false.
After your test runs, you can use the JUnit plugin to parse the results, and that will pass/fail the build.
