TestComplete: Headless Browser Testing in Parallel

In addition to standard web test execution, with the Intelligent Quality Add-on TestComplete supports running tests from an Execution Plan in a Headless browser environment. Browser options in this environment can be configured for Chrome, Edge, and Firefox, but require the installation of the supported web drivers for each of these browser types. More detailed information can be found in our support documentation on how to setup these drivers if TestComplete does not install them automatically when the environments are configured. Headless Browser Testing in TestComplete     

Here are some quick links to sites to download these web drivers as well:

 

The First Step: Running Parallel Tests in Sequence

To configure the desired testing environments in the Execution Plan, one simply needs to add the desired keyword tests or scripts to the Execution Plan, choose the Local Headless Browser option, enter the desired endpoint URL, and configure the desired test environments. It should be noted that each test in the Execution needs to be configured for the URL and the test environments individually, and that each environment can only be configured once for each test for each resolution option available (there are up to 5 resolution settings for each browser). The image below details these configuration options.

In this setup, each test in the Execution Plan will execute on the configured environments in parallel when the Execution Plan is started. The test will fire sequentially as shown here running “BearStore_Contact_Us” on the three environments simultaneously, then “Google_Panda_search” on its configured environments, and finally, “Google_Starfish_Search” on the environments configured for that test. The following figure shows the successful completion of this execution plan. Note the start times for each test showing them running in parallel, then executing the next keyword test in the Execution Plan sequence, also in parallel.

We can even up the count for each keyword test. This will result in running each test case the set number of times, also in sequence, for the given count, before moving on to the next test case.

With the resulting test log showing us those parallel test runs in sequential order:

The Next Step: Running Parallel Tests in Parallel

Now that we have run our three keyword tests in parallel environments, but in sequential order, let’s run them in parallel environments, in parallel! To accomplish this task, we need to setup a parallel Group in our Execution Plan and add our desired tests into this Group.

Then configure the Local Headless Browser URL and Environments and run the Execution Plan. Since we are now launching all of our test cases, 3 test cases x 3 configurations x 3 execution counts, TestComplete will be running 27 headless browser sessions. As you can imagine, this is extremely processor intensive. My laptop, with a few applications running, including TestComplete, hovers between 25-50% CPU utilization. Starting this test run, will easily stress my CPU at 100% for most of the duration of the test.

Our log shows all 27 tests starting at the same time. The test results also show several failures for very simple tests, most of which are associated with failures to reach the designated sites, likely caused by lack of system resources or over-taxed network connections.

In conclusion, Local Headless Browser testing can be a very useful tool for running tests in “Sequential-Parallel” or “Parallel-Parallel” modes, but system resources are a factor to consider to ensure your tests are running cleanly and successfully without generating false failures.