Test case from java secure Pro case runner has failed, but junit output looks like it has passed.

Question

Hello,&nbsp;&nbsp;One issue that i'm having is that I'm using the java approach to run the secure Pro test cases and an problem has occure.When I run the junit , certain test cases fail, but the whole junit acts as it has passed.&nbsp;I do not know how to make the junit to act as if it has failed when there are test cases that have failed.&nbsp;&nbsp;The second issue is that when I run the test from readyApi they pass, but when I run them from java case runner some of them have failed. I looked at different properties that have been set in ready api to make sure that they are uploaded to the test case runner and they have been loaded. I do not know where else to look for a solution&nbsp;Please help me with these two issues.Have a nice day,Andrei N.

andrein1 · Answer

A little update, I am using the ready api maven plugin 1.9.0 as a dependency in my pom.

I look at the difference between the test that have failed and those that pass. Aparently when I use invalid JSON types, JSON Boundary scan and JSON Fuzzy scan the test fails from the terminal but when run from the readyApi application it passes.

Still the whole junit acts as it has passed when it has failed tests.

nmrao · Answer

Would you mind showing the respective code?

andrein1 · Answer

testpublic void test() {&nbsp; &nbsp; &nbsp;SoapUIProSecurityRunner runner = new CustomSecurityRunner();&nbsp;&nbsp; &nbsp; runner.setEndpoint(endpoint);&nbsp; &nbsp;// just load the properties need to run the test&nbsp; &nbsp;setupProperties(runner);&nbsp; &nbsp;runner.setProjectFile(pathToFile);&nbsp; &nbsp;runner.setPrinReport(true);&nbsp; &nbsp;runner.run();}&nbsp;The CustomSecurityRunner extends the SoapUIProSecurityRunner and overrides the printReport() and afterRun methods.&nbsp;Hope this helps

nmrao · Answer

Thank you for the code snippet.
&nbsp;
It is mentioned that the test is passed when it is run from ReadyAPI and getting failed when it is run from Java.
&nbsp;
Now, it is difficult to comment without knowing the details of the problem i.e., why it is passed from ReadyAPI and not from Java.
&nbsp;
Based on the code snippet, can't draw any conclusion as that is very generic code which runs the project.
&nbsp;
Next to look at is, do you have any kind of assertions? what is the test case structure? how may steps and its type? Any screen shot to make easy to understand the case.
&nbsp;
&nbsp;

andrein1 · Answer

I have no assertions added to the test.As for the test case structure I have 3 test , &nbsp;two of them fail one passes.The first test that fails, has 2 steps that are rest request.Hope this is enough&nbsp;I have made another discovery, that the test that look like they have failed from java actually have not.I assume that the error comes from when I override the afterRun() method.&nbsp;here is a code snippet of the afterRun() method:&nbsp;@Overridepublic void afterRun(TestCaseRunner runner, TestCaseRunnerContext context) {super.afterRun(runner, context);...output.append("Status" + runner.getStatus().toString());...&nbsp;&nbsp;}&nbsp;A better assumption is that runner.getStatus().toString() returns fail when it should return finished.I mentioned earlier that the test which fails has the additional security scans compared to the one which passes.&nbsp;Regards,Andrei&nbsp;&nbsp;

Forum Discussion

Test case from java secure Pro case runner has failed, but junit output looks like it has passed.

6 Replies

Related Content

TestSuite Fails but all test cases are Passed

How to get TestStep Status as Passed/Failed using groovy

log4j security vulnerability

How to get pass/fail status of groovy scripting in excel sheet

Lambda integration fails quietly - with Security YAML elements

Recent Discussions

Parameterising JDBC Connection Settings..Can you do it?

Test Runner - Report All in One File

GroovyScript Help - Repeating JSON Attributes Have Specific Value.....some of the time?

JDBC results are stored in the ReadyAPI project

MyFirstPlugin - button Project menu in Ready Api