Automating Multiple Tests on Multiple Services

Question

How could this be achieved.? I have potentially a few hundred services I want to test with varying levels of fixed load. Am I right in thinking I could use a scheduler to run the test cases I have created one at a time if I carefully select the time and duration? Is this what a scheduler is for? Because I've tried to do this and nothing happens when the scheduled time comes. Also if I attached a scheduler component to a test case and save it, when I leave that project and go back in, the scheduler is no longer attached to the test case but appears to be attached to some non-existent component to the top left of the view which you cant get to. Most odd.

Also I'm a bit confused by the Load Test Automation available via the command prompt. Is this any more 'automated' than just running the test from the console?

What I'm actually after is a way of stringing together, the execution of test cases, so they run in isolation, but so that I don't have to babysit the whole suite of tests which is going to amount to thousands of test cases.

smartbear_suppo · Answer

If you want to do several tests in sequence after each other, I'd suggest taking a look at automating loadUI from Hudson (or command line), which is described in this blog post. If you're running the exact same test on different endpoints, then you probably want to use datasources/datagens via the soapUI Runner or use this component connected to a Web Page Runner.

Regarding the TestCase connector that goes up to nowhere, that seem to be a bug. Will probably be fixed today or tomorrow.

/Henrik
eviware.com

andyhughes · Answer

Well that's certainly an interesting article about the customer component connected to a SOAPUI runner. However, I'm not sure that would help me. The loadtest I have setup is designed to load the endpoint of a service (and if you want to generate any sort of load on an endpoint surely you have to make requests to the same thing rather than have a file specify which one yo want to hit.) Varying the endpoint is NOT what I want to do within the confines of this test. The idea is that this test would last for an amount of time and then a different endpoint would be tested under the same circumstances. So the ability to change the endpoint is great, but I need to do it after a certain amount of time and for separate log files to be produced for each endpoint. Is this possible?

p.s. I dont have the PRO versions (don't know if this makes any difference).

smartbear_suppo · Answer

Andy,The TestCase connector has been fixed in the upcoming nightly. However, I would be interested in seeing a screenshot from your project to maybe be able to (a) suggest a more flexible approach, or (b) improve loadUI to better suit your needs.Thanks for your bug reports!/Henrikeviware.com

andyhughes · Answer

All I have set up is a really simple test.
There is a fixed rate generator attached to a SoapUI runner, attached to a statistics component in turn connected to a table log. Thats it.
The difficulty is, there will ultimately be a few hundred endpoints for me to test, so short of cloning this test a few hundred times (and pointing the SOAPUI runner to a different place each time) I cant think of a way of doing it.
Add to this the fact that numerous fixed loads need to be tested per end point and the amount of tests balloons to a huge number which is difficult to even build never mind administer.
This is why I was looking for some way to either link the tests together so they execute one after another (not at the same time in distributed mode however, that for later down the track) with no manual intervetion and I can just pick up the data when I like OR a way of incorporating multiple tests in one, which is what I was trying to do with the URLLoader component.

If you can think of a way of doing this I'll be your friend forever.

smartbear_suppo · Answer

This answer assumes that you're familiar with soapUI property expansions, if not read about them here.

So here's what you could do:

Start loadUI with the command line option "-DmyCustomEndpoint=http://www.myWebSite.com". E.g. loadUI-cmd.bat "-DmyCustomEndpoint=http://www.myWebSite.com".

Use the property expansion ${=System.getProperty("myCustomEndpoint")} wherever you want to use your custom endpoint in soapUI.

Now you can start loadUI with a custom endpoint specified from a Bat-script, Maven, Hudson or whatever, like this:

REM testAllEndpoints.bat

loadUI-cmd.bat "-DmyCustomEndpoint=http://myWebSite.com" -p my-loadui-project.xml -r c:\dev\reports -F PDF
loadUI-cmd.bat "-DmyCustomEndpoint=http://myOtherWebSite.com/admin.php" -p my-loadui-project.xml
loadUI-cmd.bat "-DmyCustomEndpoint=http://myOtherWebSite.com"-p my-loadui-project.xml
loadUI-cmd.bat "-DmyCustomEndpoint=http://twitter.com/eviware" -p testSLA1.xml -r c:\dev\reports -F PDF

REM Add more endpoints/configurations as needed...

Regards!

/Your friend Henrik
eviware.com

EDIT: I've written a slightly more general and graphical explanation of this, here in the documentation.

Forum Discussion

Automating Multiple Tests on Multiple Services

8 Replies

Recent Discussions

Nested Loops

Bug in LoadTest context.RunCount / TotalRunCount

License/manager question

Related Content

Handling multiple selectors for same object

Multiple Test Cases

Can I put multiple project in one Git repository?