A little update, I am using the ready api maven plugin 1.9.0 as a dependency in my pom.
I look at the difference between the test that have failed and those that pass. Aparently when I use invalid JSON types, JSON Boundary scan and JSON Fuzzy scan the test fails from the terminal but when run from the readyApi application it passes.
Still the whole junit acts as it has passed when it has failed tests.
public void test() {
SoapUIProSecurityRunner runner = new CustomSecurityRunner();
runner.setEndpoint(endpoint);
// just load the properties need to run the test
setupProperties(runner);
runner.setProjectFile(pathToFile);
runner.setPrinReport(true);
runner.run();
}
The CustomSecurityRunner extends the SoapUIProSecurityRunner and overrides the printReport() and afterRun methods.
Hope this helps
Thank you for the code snippet.
It is mentioned that the test is passed when it is run from ReadyAPI and getting failed when it is run from Java.
Now, it is difficult to comment without knowing the details of the problem i.e., why it is passed from ReadyAPI and not from Java.
Based on the code snippet, can't draw any conclusion as that is very generic code which runs the project.
Next to look at is, do you have any kind of assertions? what is the test case structure? how may steps and its type? Any screen shot to make easy to understand the case.
I have no assertions added to the test.
As for the test case structure I have 3 test , two of them fail one passes.
The first test that fails, has 2 steps that are rest request.
Hope this is enough
I have made another discovery, that the test that look like they have failed from java actually have not.
I assume that the error comes from when I override the afterRun() method.
here is a code snippet of the afterRun() method:
@Override
public void afterRun(TestCaseRunner runner, TestCaseRunnerContext context) {
super.afterRun(runner, context);
...
output.append("Status" + runner.getStatus().toString());
...
}
A better assumption is that runner.getStatus().toString() returns fail when it should return finished.
I mentioned earlier that the test which fails has the additional security scans compared to the one which passes.
Regards,
Andrei
