Running tests overnight using separate capabilities
Hi all,
My job is to test a mobile device application using a long list of 23 different iPhones and 3 different iOS settings. I want to setup a sequential test plan where one phone will be run across each iOS version, then move on to the next phone, rinse and repeat. I see there is an option to do this using parallel device cloud environments but running 23x3 simulated devices all at once sounds like a task my laptop cannot handle.
Is this use-case possible in TestComplete? Is there an easy way to task runtime environments to an execution plan without coupling that test case to each individual environment? I.E. Test Case "launch app" can be run using the overnight test case runner but I can also select just that test case to run on a connected device only.
My setup is this: TestComplete v. 15.44.11x64 is linked to a Windows 10 laptop, talking to Mac Venture 13.0 Laptop with Appium 1.22.3 and XCode 14.1 running on the Mac.
Feel free to offer any advice possible. This is my first time using TestComplete (its been about a month now), so I appreciate any feedback!
Hi,
> This is my first time using TestComplete
TestComplete provides extremely great documentation, so I would recommend to get a habit to refer to documentation first in case of any question. It really worth detailed reading, especially for novices.
Back to your question and I hope that I got it right...
Are you using local device farm or cloud one like Bitbar/BrowserStack?
I just skimmed the relevant parts of TC documentation (sorry) and had not have to implement configuration like yours yet, so my rather theoretical opinion is like this:
For the local device farm you might indeed expect lack of resources from your laptop (I guess it will be Mac one). For the cloud-based farm, most probably you will have a restriction with the number of licensed parallel test executions.
Considering the above, my current idea is to consider some helper test configuration file and some helper driver code.
Configuration file might be like this:
{devices : {
1 : "iPhone 6",
2 : "iPhone 7",
...
},
iOSconfigs : {
1 : {config1},
2 : {config2},
...
}
}
And the driver code will iterate them like
for (each device in devices) {
for (each iOSconfig in iOSconfigs) {
<driver code>
}
}
And the <driver code> will craft required set of capabilities and start test execution via Parallels object provided by TC. To control the number of parallel executions you may either implement some logic inside <driver code> or try to use the Parallel Test Count option in TestComplete.
Another option that may be considered is the use of some CI/CD server.
In this case you may create either 23x3 test wrappers or 23x3 test configuration files with one test wrapper that will get required configuration as a parameter. Then you will setup a job on your CI/CD server and populate this job with 23x3 steps. Each step must be configured as running in parallel so that CI/CD server does not run them sequentially. Then you will have to configure the agent(s) that are used to execute these steps to execute not more than certain number of tasks (say 5) simultaneously. As a result, 5 tests will be started and all other will be queued. And than it will be the task for the agent to handle the queue until all queued steps are not completed.
Documentation articles:
https://support.smartbear.com/testcomplete/docs/testing-with/running/parallel.html
https://support.smartbear.com/testcomplete/docs/reference/options/engines/parallel.html
P.S. Marsha_R : Thank you a lot for pinging me 🙂
