Background: To capture test results in a data driven SoapUI groovy script based framework, we are storing the log.info steps in each script using " def logArea = com.eviware.soapui.SoapUI.logMonitor.getLogArea( "script log" ); " . This way we could store all scripts run results (taken from log.info steps). All this works perfectly fine for us and we are using this from more than a year, so no problems there.
Problem statement : Recently after reaching a stable state in project, we thought of scheduling daily health check runs using Jenkins. For this, when we integrated SoapUI with Jenkins, I found that all the steps that involve getlogArea were failing. And thus, although, I can run all scripts, which doesnt involve this step but every step that involves taking the results into a log file fails. I did some homework and searched internet to see if I can find a solution for this but couldn't find one where it solves this issue. Note that, also running soapUI direclty from command line, also have same issues.
Any pointers, solutions, workarounds are most welcome.
Thanking you in anticipation.
Hello @nmrao , Thanks for your reply. I am using the script and test status also and it works perfectly well to show what failed. But since we have a framework where the inputs from one script decide what flow will be taken in other scripts, often its the log.infos from scripts that helps us debug and troubleshoot. Also these log.infos act as test proofs of all the things we have tested (in case if we have issues in production and want to narrow down the root cause of issue). Hence the requirement. Are you aware of anything that I can use here to get these log.infos (we are writing them into text files and thats where the scripts ar failing when run from jenkins). Thanks again for taking out your time to help others. I appreciate it :).
If I were you, I would depend on the assertions rather than the logs.
Of course, the logs are useful if you want to know what happened at certain point post the execution of tests. But, can't use the logs at run time to make decisions.
In case of Data Driven tests, assuming that the data in a file, even if data of particular row / record is failed, the retest has to be happened for entire data set, and hope you agree on this.
I am using assertions very extensively in each script. Neither am I using logs to make decisions. I also agree with you that if a failure happens, the retest has to happen for full data set. But that's not why I (or my team) is entirely using the logs.
We are using logs for two purposes. As test results and to look back and understand what all paths are taken in previous scripts (if there is a failure). There are too many parameters that we have to consider during debugging. So we use individual scripts status, overall test status, assertions, logs all (and each serve a particular purpose).
The problem now is when I want to schedule health checks, all test steps where we are using logarea are failing. Is there is a solution (or workaround) that we can use?
Thanking you once again for your time and support.
I am sorry. Tried understand your case from your question and replies, but unable to do so. Could not offer any help in this regard.
However, one input I have is that there are certain API / methods work in only SoapUI and they do not work out side of UI.
For instance, object "workspace" perfectly works fine from UI and doesn't work if "testrunner" script is used.
So, check if your tests fall into same category (without even going to Jenkins) by running the project with testrunner (And I hope you are using this utility from Jenkins as well).
No apologies, I understand you are only trying to help. I am indeed using testrunner.bat in jenkins in its command line option . And irrespective of if I use it from jenkins or directly from windows command line, the results are same. So may be you are right that this method is only available from UI. Is there a place you are aware where I can find a list of objects that are only available from UI (Or not available from command line?).
@TanyaGorbunova : Or if SoapUI creators are listening, making this possible with command line?
Thanks once again for all your help, support and time. I do appreciate it a lot. Keep up the good work :).