TNeuschwanger-Thanks for the response!
At this point I have figured out how to do what I wanted in terms of being able to loop through all the tests in the suite one at a time, then through each test step in each case, run dynamic data retrieval SQL's and use the values for checkpoints, BUT I am still trying to figure some things out - especially the use of assertions if they fail in the groovy script (not in the embedded groovy script of a step mind you - just in the Groovy Script test that acts as my driver). If the assertion fails the script errors out - that doesn't seem to make sense as I thought it would simply cause the test to fail - so I built my own code to do the check at this point. The assert seems to have made itself useless because of this fail possibility.
Also, using the messageExchange object seems to only be available if you are in a Script Assertion step within a test step - I don't know what I need to import in a regular Groovy script to have access to that so I worked my way around it. I get an error when I try to use messageExchange so...not sure what to do other than not use it here. I can use it in the embedded Assert Script but that defeats the purpose of having ONE SCRIPT for all test cases.
The point of what I am trying to do is make it so that all the guys on my team, WHO HAVE NEVER DONE API testing or used this tool or written Groovy scripts, can create test cases simply through an excel spreadsheet. I will (eventually) read from an Excel file what the desired test is to create and dynamically create it - assertions and all, dynamically through the driver script - much like I currently do with our functional testing in UFT.
I just started with SoapUI - in my free time for the last month or so - and found it cumbersome (the free version mind you as work is too cheap to pay for anything) to build a single test case manually - you have to get the wsdl, make all the header settings, set up properties, do data prep with the AUT to get to initial condition, then make your assertions for each step one by one within the tool, not to mention the ridiculously poor debugging tools and meaningless error messages that tell you the error is on line 12 when it's actually on line 378 etc. Also, am I missing capabilities like putting breakpoints and viewing runtime variable values while the script is in progress? If so, please advise as I am all ears and hoping it gets better. So anyway, I am writing a script that runs in whichever test suite you place it, and utilizes the various property levels to make as much of the testing dynamic with the focus being FUNCTIONAL testing.
My big problem right now is figuring out how to make sure the test steps use the property values I put in the api variable (parameter) places - for example, I think I mentioned above, each of my tests is to be written with a SQL statement that finds a valid piece of data that meets needed initial conditions - writes the result to a test case property, and that test case property is referenced in the parameter space for the API call - when the script runs - it fails - don't know why as log doesn't tell me. BUT, after the run, since the data has been retrieved and written to the testcase property being referenced, if I click Run manually to run it again - it works and uses the value the SQL retrieved. So..what's the deal there?
I will be the first to admit I am NOT a Java / Java Script / object oriented programming languages person. I am self taught at everything I do and as such - what I've learned MAY BE WRONG! So if you have some advice -I will take it.
Also, in regards to the logs that SoapUi provides - they suck. Maybe I'm missing something but they simply tell you whether or not the test step passed or failed - but lack the details as to WHY. For example, I created 5 test cases - ran them at the Test Suite level and generated the report. The only info given is the basics and the response package BUT there was nothing about which assertion failed even for the assertions I created with the built-in tool. If I have 5 or 6 different checks I want to know WHICH one failed IN THE REPORT LOG so I don't have to go open up the test steps individually and see which are green and which are red...what am I missing? Again - I have written my own checkpoints now and don't need the built in ones unless I want to use them so I can turn the step green or red and make it look pretty that way. Seems to me, if you grab the response and convert it to a string you can easily do almost ALL the checks that are "built-in" with very little effort using the Groovy commands - ie contains, count etc. The manual creation part is very cumbersome and time consuming.
Let me know what you think - especially on the parts where I have actual questions and not so much the complaints - I complain alot for a non-developer type person - I know that. But hey, with a lack of knowledge comes a willingness to blame others!
hello bqualters
That is a lot to unpack for a question you need answered... đ
"If the assertion fails the script errors out - that doesn't seem to make sense as I thought it would simply cause the test to fail"
If you would like the testcase to finish even though an assertion fails you can un-ticked the "Abort test if an error occurs" within the TestCase Options feature of every testcase.
Also, in regards to the logs that SoapUi provides - they suck. Maybe I'm missing something but they simply tell you whether or not the test step passed or failed - but lack the details as to WHY
You get as much out of the log information as you put into them... My testcases consist almost entirely of Groovy test steps...
As you can see from the sample Groovy I submitted before, you can put as much log information as you need to help debug your scripts. As a testcase runs, I can view the "script log" tab at the bottom of SoapUI screen to see all of the log statement content so I know exactly where my issue might be when a failure occurs. I often include a lot of log.info code when developing a test and comment it out or delete it after I see what is happening. If there is a groovy failure, I can usually review the "error log" tab for specific language/programmatic failure reasons. That might be a communication gap here... I use the script log as my canonical report which is different than an actual report that you can generate in SoapUI user interface.
I was drawn to SoapUI to begin with because of its ability to be flexible in meeting my needs. If it did not have a feature I needed, I could write my own in Groovy. I think I won't be of much help in your endeavor to create a monolithic script to do what you want. I AM a Java / Java Script / object oriented programming languages person and prefer my projects to be modular in nature instead of your desired "purpose of having ONE SCRIPT for all test cases." I can try to answer small specific questions though.
Regards,
Todd
Still not sure why this is happening-->
I can not seem to get the test steps to execute using the referenced variables (either as part of the request or part of the assertion) - for example, I run a SQL - it retrieves an OrderKey - which I write as a Property at the testcase level - then I sub in that value using the parameter in the API Request of the test step and instead of using the hardcoded value I supply ${#TestCase#OrderKey} in the parameter value column - this works if I trigger the execution of the step manually as it uses the value currently stored in the test case Property called OrderKey BUT--> when I run the script to get it to execute this step it fails even though the SQL has run and defined the value in the test case property (OrderKey) successfully - it fails to make the request and throws an error. If I manually execute it again at this point with the retrieved value from the SQL - it works. The same goes for the Assertions - if I reference a property - it fails when run through the script (which I removed the variablized "submission" value and hardcoded it) - but if I click the Request to run manually with same setup - it works.
Sooo...what's the deal?
hello bqualters
it is hard to visualize what you are doing... could you include some screen pictures of your test case and show the code/dialog where you are setting and getting properties? Are you using jdbc, property transfers, groovy script for the tasks you describe?
The community edition of SoapUI is a pretty mature product, so I don't think it could be a bug or many people would have encountered this issue. We will have to look to see what the test steps are in your test case.
Regards,
Todd
I was using code in a way to set the property variable but - I thought if I put ${#TestCase#HexKey} for example in the Request itself that it would automatically use whatever value was stored in the TestCase property referenced. The value may change from one run to the next as I run an SQL to retrieve a valid value from the database and write that value as the HexKey testcase property BEFORE having the Groovy script driver call the test step to execute. The Request works if I manually trigger it's execution (as the value has already been written to that property from previous script attempt to run it where the SQL gets the value - it doesn't go away) but if I run the script it fails and returns-->{"Message": "No HTTP resource was found that matches the request URI 'http://api.xxxxx.xxxxx1.xxx.sc.xxx.com/xxxxx
/createauditbycase/'."} (I replaced parts of the URL with xxx's obviously) -
Below is a pictur eof how I have the request set up - works manually -(The Not Found message IS EXPECTED as using bad value) but try to run it with a command from Groovy and get the above error mentioned. It looks like it sent a null value I guess. I execut eit in the script with these commands-->
status = tstep.run(testRunner,context);
result = status.getStatus().toString();
If I hard code the value in the Value column pictured it works both ways - manually and through the script. This is obviously not good so I need to understand what is wrong.
