It sounds to me like this is a candidate for consolidating everything into a single test that is parameterized based upon a table. Rather than having hundreds of test items, you could have one test item with a parameter the runs an SQL query on a CSV file to pull out a subset of tests. Consider a minor redesign of your project on those lines as I'm not sure how to run based upon a group.
The other option would be to create multiple "parent" test items as your group place holders, have them log a simple bit of "This is group XXXX" and then have the appropriate test items attached as child items under that. You should then be able to run the parent and the child ones will execute. Personally, this feels like a lot of work that can be handled by simplifying the whole project.
Hi Robert,
In the Jobs list, I don't see a way to define a job to reference an individual test item either, or to pass a parameter to one.
(where 'test item' is defined as an those in the Test Items tab of the Project).
In the Test lookup, the only thing I can find are procedures in the project.
To make that work, I would have to define a different procedure for each group, that internally calls the list of tests (csv files) in that group. That's certainly possible if it's the only way to do it, but would involve some structural rework, especially since I have some custom code in GeneralEvents_OnStopTest that relies on Project.TestItems references.
If there was a way to launch Test Items directly from scripts I could define a procedure for each group, but that doesn't seem to be supported.
Still thinking...
Allen
How about a single slave project that includes a routine or something that first of all queries the machine name and then, based upon the machine name, executes a particular set of tests? Yes, this would take some restructuring, but I think it would, over all, be less complicated. What you would do, then, is have the main routine of the project use a DDT object that would execute an SQL query against your CSV file, that SQL query altered based upon the machine name. So, Machine1 would get all tests where GroupName = "Machine1", Machine2 would get all tests where GroupName = "Machine2" and so on.
Execute that slave project on your slave machines.. they are now context-aware..., they know what machine they are running on and will then execute any tests you need to. What this would mean is that the slave project, instead of having a whole list of test items, would ahve a single test item that would call your test, parameterized based upon machine name.
Given time, I could write up the code for you but, perhaps, this will trigger some thoughts so you can fiddle with it yourself?
Hi Robert,
I understand your proposed solution, and it is something that I could implement myself.
Thinking through some possible cons:
Test groupings would be somewhat harder to visualize and maintain, most likely maintained in Excel instead of natively.
Built in timeout capability for individual tests would be lost. I would need to implement my own if needed (rare, but it does happen).
So far nothing insurmountable, just a little work.
Error logging: That's a problem. How could test failures be broken out and reported for individual tests out of a group?
Another possibility would be to implement keyword tests to call my CSV launcher script or a group driver script directly, just like a test item. I haven't used them before, but they are selectable individually for a distributed project. Seems a bit of a kludge, but might maintain some of the capability above.
FYI: I did query SmartBear support directly and got the official answer to my initial query:
"You cannot select test items or an item group to run on a slave machine. As a test in task properties, you can choose a keyword test or a function in a script unit:
https://support.smartbear.com/viewarticle/68430/"
Anyway, my original question is answered (even if I don't like the answer) and I have some options now, either pure code + CSV, keyword tests, or some hybrid approach. I'll play around for a while and see which one works out.
Thanks!
