Recent Content
Azure DevOps Pipelines - Running “Headless” Tests
One of the most common requests I get from TestComplete customers who run their tests from Azure DevOps pipelines is: “How can I execute my tests on remote Virtual Machines (VMs) with self-hosted agents without needing to maintain an active terminal session?”. Note: This document does not detail setting up the Azure DevOps pipeline or the self-hosted Azure agents. That process can be found in the TestComplete documentation here: Integration With Azure DevOps and Team Foundation Server via TestComplete Test Adapter | TestComplete Documentation There are some details in the TestComplete documentation on how to accomplish this type of configuration, but I’ll go through the full process and how to configure this solution with two options. That documentation can be found online: Disconnecting From Remote Desktop While Running Automated Tests | TestComplete Documentation The first option is to set up multiple virtual machines (VMs). You start by logging into one VM via Remote Desktop Protocol (RDP). From there, you connect to other Tester VMs running Microsoft Agents. After issuing a command to release the RDP session back to the Admin user, the Tester VMs remain active, allowing TestExecute to run tests triggered by the pipeline. The second option is to use a single, high-performance VM to host all remote sessions and agents. Here, multiple Tester sessions run on the same VM, each with its own agent handling pipeline requests. Each session runs its own TestExecute instance as long as enough floating licenses are available. The Admin session connects to the VM and then accesses each Tester session via RDP using the VM’s loopback address (e.g., 127.0.0.2). However, this method requires the VM to be part of a Microsoft Domain to support more than two simultaneous RDP sessions. Here are the commands we’ll be taking advantage of in this operation. To gracefully disconnect from an active RDP terminal session, converting it into a console session, open a Command Prompt (CMD) in the Admin user session and use the following: %windir%\System32\tscon.exe RDP-Tcp#NNN /dest:console where RDP-Tcp#NNN is the ID of your current Remote Desktop session, for example, RDP-Tcp#5. You can see it in the Windows Task Manager on the Users tab, in the Session column. The Session column is hidden by default. To show it, right-click somewhere within the row that displays CPU, Memory, and so on, then choose Session in the opened context menu. Our documentation also includes an easy-to-use batch file option that can be run as an Administrator to more easily disconnect from the remote Admin session. Create a batch file with this code: for /f "skip=1 tokens=3" %%s in ('query user %USERNAME%') do ( %windir%\System32\tscon.exe %%s /dest:console ) Create a desktop shortcut to this file. To do this, right-click the batch file and select Send to > Desktop (create shortcut). In the shortcut properties, click Advanced and select Run as administrator. Double-click this shortcut on the remote computer (in the Remote Desktop Connection window) or call this batch file at the beginning of your tests (provided that the tests are run as an Administrator). To gracefully reconnect to a remote console session, use the following command in an open CMD prompt: mstsc -v: servername /F -console where the servername is the address of the remote VM hosting the Admin session. Option 1: Multiple VMs in the testing network. RDP connect to and set up the remote VMs to be used for the Tester Agents and have their self-hosted Microsoft agents active and listening for jobs, then disconnect the RDP sessions. RDP connect to the VM to be used as the “Admin” session. From the “Admin” VM, RDP connect to each Tester VM. (Connecting to more than 2 remote RDP sessions will require the VMs to be on a Domain with the appropriate settings configured to allow for more than 2 remote sessions.) Your Admin system should look something like this image where I am connected to 2 of my Tester VMs with their agents listening for jobs: Ensure that your Agents are active in the Azure DevOps Agent pool: Disconnect from the Admin user session using the prepared batch file or the direct command with the RDP session ID: Run the pipeline to validate that tests execute as expected: Reconnect to the Admin VM to see test results from the running agents using the command: mstsc -v: servername /F -console Option 2: Using a Single Virtual Machine RDP connect to the VM to be used for testing using the “Admin” account. I’m using my Tester01 account in this demonstration. RDP connect to the other users on the same VM using the 127.0.0.2 loopback address. Run the Microsoft self-hosted agents on these connected sessions. Disconnect from the Admin user session using the prepared batch file or the direct command with the RDP session ID. Prepare the pipeline. In my case, as I am not on a Domain so I am limited to ONE RDP session from my “Admin” user. I’m disabling the agent on Tester03 since we are using Tester02 for our pipeline. Execute the pipeline and validate successful test executions: Reconnect to the “Admin” session to make changes to the configuration as needed: mstsc -v: servername /F -console Conclusion In conclusion, these two methods provide testing options from Azure DevOps using VMs without requiring a dedicated terminal session from your development system. They still require that session, but the solutions provide options to workaround that “directly monitored” connection. These solutions also give you the security of running them on VMs in your environment; easily secured, controlled, and isolated by your firewall and security requirements. Full demonstration video link: Azure DevOps Pipelines - Remote "Headless" Testing1like0CommentsData Structures for Application Object Definitions
Library files containing data structure objects like JavaScript objects, Map objects, and JSON objects work well across multiple projects and suites. These files are also nice to work with in source control. Changes in any library file can be easily seen and tracked. Each of these object types will integrate seamlessly with methods like FindChild and work equally well with desktop and web applications. JavaScript objects are directly usable in TestComplete scripts. Map objects have some advantages over JavaScript objects such as built in looping methods and properties. This adds some complexity in implementation as these methods must be used to access the data structure. JSON is a 'portable' object definition. Many different languages support JSON. JSON requires a parse operation to return a JavaScript object prior to use in JavaScript. JSON would be preferred if the object definitions were created directly from the application code base by development. Object Definition Examples This code defines an object named customOptionObjDefs. It contains two UI object definitions: btnOK and btnCancel (It could contain many definitions). The code below shows examples of a JavaScript Object, a JavaScript Map and a JSON Object and the code used with the TestComplete method FindChild. JavaScript Object const customOptionObjDefs = { btnOK: { propertyNames:["WPFControlName","WPFControlText"] ,propertyValues: ["btnOK","OK"], depth: 16}, btnCancel: { propertyNames:["WPFControlName","WPFControlText"] ,propertyValues: ["btnCancel","Cancel"], depth: 16} } const btnOKDef = customOptionObjDefs.btnOK; // directly accessable const btnOKObject = parentObject.FindChild(btnOKDef.propertyNames,btnOKDef.propertyValues,btnOKDef.depth); JavaScript Map Object const customOptionObjDefs = new Map([ ["btnOK", { propertyNames: ["WPFControlName", "WPFControlText"], propertyValues: ["btnOK", "OK"], depth: 16 }], ["btnCancel", { propertyNames: ["WPFControlName", "WPFControlText"], propertyValues: ["btnCancel", "Cancel"], depth: 16 }] ]); const btnOKDef = customOptionObjDefs.get("btnOK"); // .get method const btnOKObject = parentObject.FindChild(btnOKDef.propertyNames,btnOKDef.propertyValues,btnOKDef.depth); JSON Object const customOptionObjDefs = { "btnOK": { "propertyNames": ["WPFControlName", "WPFControlText"], "propertyValues": ["btnOK", "OK"], "depth": 16 }, "btnCancel": { "propertyNames": ["WPFControlName", "WPFControlText"], "propertyValues": ["btnCancel", "Cancel"], "depth": 16 } }; const btnOKDef = JSON.parse(customOptionObjDefs).btnOKDef; // .parse method const btnOKObject = parentObject.FindChild(btnOK.propertyNames,btnOK.propertyValues,btnOK.depth); I prefer to organize object definition libraries in files by application and form and 'importing them in scripts or in the class structure of a project using 'require'. These files can be stored externally to the project, shared and organized as desired. const orderEntryDefs = require("orderEntryDefinitions"); Sometimes application objects are not well named or require a variable to be calculated dynamically in order to be defined. These objects are a challenge to define for automation and usually lead to brittle code. In such cases code is written to determine the values needed for definition and passed directly to the FindChild method or a 'helper' class method or function. In most cases a helper class method or a function is used to create and return the objects defined in the definition libraries. The use of a 'helper' class method or function will provide a layer of abstraction, centralized error handling and more modular code. The file(s) containing these functions or methods would also be imported for each script using 'require'. Conclusion The use of data structures provides a modular way to define and create instances of objects for automation scripts. These objects can be easily looped over to find and create container objects for entire forms or very complex object like grids or trees. I have found that creating all available objects for a form or in groups if the form changes dynamically and storing them in a object makes the script easier to write and to read. WW Wood Products Inc.0likes0CommentsHow To: Read data from the Windows Registry
Hello all, I have recently learned how to retrieve data from the Windows registry in JavaScript test units. I am using this to return the OS information and application path information. This is very useful when added to the EventControl_OnStartTest event code. This will allow you to return OS information and other needed data at each test run. Some test management systems may provide this information for you or it may be logged in the in data produced in a pipeline run. This will embed the information directly into your test log. SmartBear KB Links: Storages Object Storages Object Methods Storages.Registry Method Section Object Get SubSection Method This bit of code will return the Product Name and Current Build from the registry. This location may vary between OS's so you will want to check this with RegEdit. let Section = Storages.Registry("SOFTWARE\\Microsoft\\Windows NT", HKEY_LOCAL_MACHINE); let regKeyString = Section.GetSubSection("CurrentVersion").Name; let productIdString = Storages.Registry(regKeyString, HKEY_LOCAL_MACHINE, 1, true).GetOption("ProductName", ""); let currentBuildString = Storages.Registry(regKeyString, HKEY_LOCAL_MACHINE, 1, true).GetOption("CurrentBuild", ""); Log.Message("Windows Version: " + productIdString + " Build: " + currentBuildString ) I have also found the need to find and set an application path and work folder in the project TestedApp for running through a pipeline because the pipeline deploys the application to a non-standard path. let Section = Storages.Registry("SOFTWARE\\WOW6432Node\\<_yourSectionName>\\", HKEY_LOCAL_MACHINE); let regKey = Section.GetSubSection(<_yourSubSectionName>).Name; let Path = Storages.Registry(regKey, HKEY_LOCAL_MACHINE, 0, true).GetOption("", ""); let WorkFolder = Storages.Registry(regKey, HKEY_LOCAL_MACHINE, 0, true).GetOption("Path", ""); let appIndex = TestedApps.Find(<_yourAppName>); if (appIndex >= 0){ if(TestedApps.Items(<_yourAppName>).Path != Path){ TestedApps.Items(<_yourAppName>).Path = Path } if(TestedApps.Items(<_yourAppName>).WorkFolder != WorkFolder){ TestedApps.Items(<_yourAppName>).Params.ActiveParams.WorkFolder = WorkFolder; } } else{ Log.Error("TestedApp " + <_yourAppName> + " does not Exist.") Runner.Stop(true); } I hope you find these links and code examples as useful as I have! Have a great day!Drag-and-drop object to another object
TestComplete's built-in Drag action is designed to drag a specific Alias object from a given point to another point, but at a pixel offset (i.e. drag Alias....Button by X/Y pixels). While useful as a "jumping off point", this approach can be problematic for obvious reasons (Dynamic UIs, changing screen resolutions, inconsistent offsets) leading to brittle tests. Fortunately, TestComplete method parameters offer a high degree of customisation. By evaluating and utilising exposed properties like ScreenTop/ScreenLeft, we can create more robust and adaptable drag-and-drop actions. This allows us to instead dynamically reference the coordinates of a target object, or better still use exposed values in simple calculations, like figuring out the offset value for the Drag action. This Python script example calculates the offset using the common ScreenTop and ScreenLeft positions of both objects then passes the difference to the Drag action, allowing us to drag one given object to another given object with much more flexibility : def dragToObject(clientObj, targetObj): # Using ScreenLeft property to drag horizontally; ScreenTop for vertical fromObjectTop = aqObject.GetPropertyValue(clientObj, "ScreenTop") fromObjectLeft = aqObject.GetPropertyValue(clientObj, "ScreenLeft") toObjectTop = aqObject.GetPropertyValue(targetObj, "ScreenTop") toObjectLeft = aqObject.GetPropertyValue(targetObj, "ScreenLeft") dragY = toObjectTop-fromObjectTop dragX = toObjectLeft-fromObjectLeft Log.Message("Dragging "+aqConvert.IntToStr(dragX)+"px horizontally and"+aqConvert.IntToStr(dragY)+"px vertically") clientObj.Drag(-1, -1, dragX, dragY) You can then even utilise this in your KeywordTests, by changing the input parameter Mode to Onscreen Object, which enables the Object Picker : Now you have a way to drag one object to another - for example a value into a table? Hope this gets the creative juices going - can you think of other ways you might handle dynamic values in other Action methods? Regards, Mike TestComplete Solutions Engineer0likes0CommentsClick Object not on screen
I have an application in .Net that looks like the old Windows 8 Metro interface (lots of "tiles" with a horizontal scroll bar). Using the object spy and aliases, I was able to find my "tile" and create a script to click it. As long as the tile is on the screen (scrolling left or right to move my tiles), it works. However, as soon as that tile is not on the screen, it fails to find it even though it is mapped. I am not a developer but trying to learn. Any ideas on how to make the code click the object? Aliases.MYCustomAPP.Tile_WO.Click(); The full path of the object is: Sys.Process("MYCustomAPP").WPFObject("HwndSource: DynamicWindow", "MYCustomAPP : test").WPFObject("DynamicWindow", "MYCustomAPP : test", 1).WPFObject("LayoutRoot").WPFObject("radTileList").WPFObject("TileGroupContainer", "Production", 7).WPFObject("Tile", "", 1).WPFObject("Grid", "", 1).WPFObject("TextBlock", "Work Order", 1)0likes0CommentsTestComplete: Headless Browser Testing in Parallel
In addition to standard web test execution, with the Intelligent Quality Add-on TestComplete supports running tests from an Execution Plan in a Headless browser environment. Browser options in this environment can be configured for Chrome, Edge, and Firefox, but require the installation of the supported web drivers for each of these browser types. More detailed information can be found in our support documentation on how to setup these drivers if TestComplete does not install them automatically when the environments are configured. Headless Browser Testing in TestComplete Here are some quick links to sites to download these web drivers as well: Chrome Edge Firefox The First Step: Running Parallel Tests in Sequence To configure the desired testing environments in the Execution Plan, one simply needs to add the desired keyword tests or scripts to the Execution Plan, choose the Local Headless Browser option, enter the desired endpoint URL, and configure the desired test environments. It should be noted that each test in the Execution needs to be configured for the URL and the test environments individually, and that each environment can only be configured once for each test for each resolution option available (there are up to 5 resolution settings for each browser). The image below details these configuration options. In this setup, each test in the Execution Plan will execute on the configured environments in parallel when the Execution Plan is started. The test will fire sequentially as shown here running “BearStore_Contact_Us” on the three environments simultaneously, then “Google_Panda_search” on its configured environments, and finally, “Google_Starfish_Search” on the environments configured for that test. The following figure shows the successful completion of this execution plan. Note the start times for each test showing them running in parallel, then executing the next keyword test in the Execution Plan sequence, also in parallel. We can even up the count for each keyword test. This will result in running each test case the set number of times, also in sequence, for the given count, before moving on to the next test case. With the resulting test log showing us those parallel test runs in sequential order: The Next Step: Running Parallel Tests in Parallel Now that we have run our three keyword tests in parallel environments, but in sequential order, let’s run them in parallel environments, in parallel! To accomplish this task, we need to setup a parallel Group in our Execution Plan and add our desired tests into this Group. Then configure the Local Headless Browser URL and Environments and run the Execution Plan. Since we are now launching all of our test cases, 3 test cases x 3 configurations x 3 execution counts, TestComplete will be running 27 headless browser sessions. As you can imagine, this is extremely processor intensive. My laptop, with a few applications running, including TestComplete, hovers between 25-50% CPU utilization. Starting this test run, will easily stress my CPU at 100% for most of the duration of the test. Our log shows all 27 tests starting at the same time. The test results also show several failures for very simple tests, most of which are associated with failures to reach the designated sites, likely caused by lack of system resources or over-taxed network connections. In conclusion, Local Headless Browser testing can be a very useful tool for running tests in “Sequential-Parallel” or “Parallel-Parallel” modes, but system resources are a factor to consider to ensure your tests are running cleanly and successfully without generating false failures.1like0CommentsTestComplete with Zephyr Scale
In this post, we are going to talk about SmartBear’s UI testing tool, TestComplete, and writing the results to Zephyr Scale in an automated fashion. When we think about using TestComplete with any test management tool, it can really be accomplished in two ways: Natively inside TestComplete or integrating with some CI-CD system. When we are using Zephyr Scale, both ways will utilize the Zephyr Scale REST API. When we link to Zephyr Scale natively from TestComplete, it is a script heavy approach. An example of that can be found here. Today we are going to go into detail about using TestComplete, with a CI-CD system, and sending the results to Zephyr Scale. Now let’s talk about automating this process. The most common way to automate triggering TestComplete tests is through one if its many integrations. TestComplete can integrate to any CI-CD system as it has Command Line options, REST API options, and many native integrations like Azure Dev Ops or Jenkins. The use of a CI-CD system makes managing the executions at scale much easier. The general approach to this workflow would be a two-stage pipeline something like: node { stage('Run UI Tests') { // Run the UI Tests using TestComplete stage('Pass Results') { //Pass Results to Zephyr Scale } } First, we trigger TestComplete to execute our tests somewhere. This could be a local machine or a cloud computer, anywhere, and we store the test results in a relative location. Next, we use a batch file (or alike) to take the results from that relative location, send them to Zephyr Scale. When executing TestComplete tests, there are easy ways to write the results to a specific location in an automated fashion. We will look at options through the CLI as well as what some of the native integrations offer. Starting with the TestComplete CLI, the /ExportSummary:File_Name flag will generate a summary report for the test runs, and save it to a fully qualified or relative path in Junit-XML structure. At a basic level we need this: TestComplete.exe <ProjectSuite Location> [optional-arguments] So something like this: TestComplete.exe "C:\Work\My Projects\MySuite.pjs" /r /ExportSummary:C:\Reports\Results.xml /e The /ExportSummary flag can be stored in a relative or fully qualified directory. We could also use one of TestComplete’s many native integrations, like Jenkins and specify in the settings where to output results: Now that our TestComplete tests are executing, and the results are writing to a relative location we are ready for stage 2 of the pipeline, sending the results to Zephyr Scale. So now let’s send our results to Zephyr Scale. I think the easiest option is to use the Zephyr Scale API, and the Auto-Create Test Case option to true. The command below is a replica of what you would use in a batch file script in the pipeline. curl -H "Authorization: Bearer Zephyr-Scale-Token-Here" -F file= Relative-Location-of-Report-Here\report.xml;type=application/xml "https://api.zephyrscale.smartbear.com/v2/automations/executions/junit?projectKey=Project-Key-Here&autoCreateTestCases=true" After you modify the API token, relative location, and project key you are good to run the pipeline. The pipeline should look something like this: After we run the pipeline let’s jump into Jira to find to confirm the results are populating. Even with execution data: Also, with transactional data to analyze the failed test steps:1like0CommentsCrossBrowserTesting to BitBar Selenium Script Migration - QuickStart Guide
On June 21, 2022, SmartBear launched web application testing on our unified cloud testing solution that will include both browser and device testing on the BitBar platform! We have listened to our customers and having one product for both web and device testing will better meet your needs. BitBar is a scalable, highly reliable and performant platform with multiple datacenters. On it, you will have access to the latest browsers and devices with additional deployment options to meet your needs, including private cloud and dedicated devices. For more frequently asked questions about the launch of web app testing on BitBar, visit our FAQ. This Quickstart Guide is intended to walk through the conversion of your existing CrossBrowserTesting Selenium tests to use BitBar! We have updated Selenium hubs and API calls that will require conversion, though little else will be required. As with CrossBrowserTesting, we have sample scripts and a Selenium Capabilities Configurator you may use to build out the specific capabilities for the desired tested device. This tool can be found here. To start conversion, you will need your BitBar API key versus the CrossBrowserTesting Authkey. This is a new method to authenticate the user and to make API calls. You may find your BitBar API Key in account settings as described here. Most of the Code examples and talking points for conversion are in reference to the CrossBrowserTesting Selenium Sample script that is available here. All code snippets in this article will be in Python. Now that you have your BitBar API Key, let's alter the original Authkey variable with our new BitBar API key located at line 18 in the CrossBrowserTesting sample script. This step is for connection to the BitBar API for processes such as taking screenshots and setting the status of your tests. # Old CrossBrowser Testing Sample Authkey Variable self.authkey = "<"CrossBrowserTesting Authkey">" # New Bitbar API Key Variable self.apiKey = "<"insert your BitBar API Key here">" In regards to the capabilities used in BitBar, there are a couple things to note. First, we do not need to specify a 'record_video' capability as we do in CrossBrowserTesting. Videos are generated automatically for every test, so we no longer need to provide this capability. Doing so will result in webDriver errors. The second thing to note is that we now also pass the BitBar API Key along with the Capabilities; capabilities = { 'platform': 'Windows', 'osVersion': '11', 'browserName': 'chrome', 'version': '102', 'resolution': '1920x1080', 'bitbar_apiKey': '<insert your BitBar API key here>', } With BitBar we now have four Selenium hub options to choose from. Both US and EU Selenium hubs are available to aid in performance for your location. Separate hubs are also provided depending on the type of device (Desktop vs Mobile) you wish to test against. You may pick the applicable Desktop or Mobile hub closest to your location and replace your existing hub with the updated URL; BitBar Desktop Selenium Hubs; US_WEST:DESKTOP: https://us-west-desktop-hub.bitbar.com/wd/hub EU:DESKTOP: https://eu-desktop-hub.bitbar.com/wd/hub BitBar Mobile Selenium Hubs; US_WEST:MOBILE: https://us-west-mobile-hub.bitbar.com/wd/hub EU:MOBILE: https://eu-mobile-hub.bitbar.com/wd/hub # start the remote browser on our server self.driver = webdriver.Remote desired_capabilities=capabilities command_executor="https://us-west-desktop-hub.bitbar.com/wd/hub" Now that we have our BitBar API Key, and Capabilities and Selenium Hub set up, we can move on to altering our requests for Screenshots and Test Result Status. In the CrossBrowserTesting sample script, we use standalone API requests to create Screenshots. For the BitBar sample scripts, we are doing this with the Selenium driver itself to create the Screenshot and store it locally. Afterwards use the BitBar API to push the locally saved image back to our project. The swagger spec for our BitBar Cloud API can be found here. In line 30 of the BitBar Selenium sample script we set a location to store Screenshots on the local machine. Note, this is set up to store files in a directory called 'Screenshots' in the root folder of your project. self.screenshot_dir = os.getcwd() + '/screenshots' To retrieve a Screenshot and store it, we perform a 'get_screenshot_as_file' call, as seen on line 45 in the BitBar Selenium example script. self.driver.get_screenshot_as_file(self.screenshot_dir + '/' + '1_home_page.png') Now we want to to take our Screenshot and push it back to our project in BitBar. Note that in this case for Python, we are using the 'httpx' module for the API calls back to BitBar. The 'requests' module only supports HTTP 1.1 and we will need a module capable of handling HTTP 2/3 requests. # Let's take our locally saved screenshot and push it back to BitBar! # First we start by declaring the 'params' and 'files' variables to hold our Screenshot name and location. params = { 'name': self.screenshotName1, } files = { 'file': open(self.screenshot_dir + '/' + self.screenshotName1, 'rb'), } # Now we build out our API call to push our locally saved screenshot back to our BitBar Project print("Uploading our Screenshot") response = httpx.post('https://cloud.bitbar.com/api/v2/me/projects/' + self.projectID + '/runs/' + self.RunId + '/device-sessions/' + self.deviceRunID + '/output-file-set/files', params=params, files=files, auth=(self.apiKey, '')) # Here we check that our upload was successfull if response.status_code == 201: print("Screenshot Uploaded Successfully") else: print("Whoops, something went wrong uploading the screenshot.") The final piece of the puzzle is to set our Test Result Status. We have alternate naming conventions for test results, these are 'Succeeded' and 'Failed' for BitBar vs 'Pass' and 'Fail' for CrossBrowserTesting. # CrossBrowserTesting Successful test syntax self.test_result = 'pass' # CrossBrowserTesting Failed test syntax self.test_result = 'fail' # BitBar Successful test syntax self.test_result = 'SUCCEEDED' # BitBar Failed test syntax self.test_result = 'FAILED' Note that in the snippet provided below, we start by performing Get requests for session information. These requests are sent to the same Selenium Hub we are using for the webDriver, so make sure the hub address is set to the same hub used for webDriver. We would recommend to turn this into a variable to avoid having to switch this manually for alternate hubs. These processes are found in the 'tearDown' function of the updated CrossBrowserTesting sample script found here. #get all necessary IDs of current session response = requests.get('https://us-west-desktop-hub.bitbar.com/sessions/' + self.driver.session_id, auth=(self.apiKey, '')).json() deviceRunID = str(response["deviceRunId"]) projectID = str(response["projectId"]) RunId = str(response["testRunId"]) Finally, we set the Test Result with the Post method below using session information retrieved with the Get request above. Note, the URL for the Post request will NOT need to be updated to reflect the specific Selenium hub in use. # Here we make the api call to set the test's score requests.post('https://cloud.bitbar.com/api/v2/me/projects/' + projectID + '/runs/' + RunId + '/device-sessions/' + deviceRunID, params={'state': self.test_result}, auth=(self.apiKey, '')) Now that we have made these changes you are ready to run your test through BitBar! As a summary, we replace our CrossBrowserTesting authKey with the BitBar API Key, set the new Selenium hub address, build new screenshot calls and update the test result function. Quick Reference Documentation; BitBar Web FAQ. Complete documentation with code samples in various languages are found here. Retrieve your BitBar API Key in account settings as described here. BitBar Selenium Capability Configurator and Sample Scripts are found here. CrossBrowserTesting Capability Configurator and Sample Scripts are found here. The Swagger spec for our BitBar Cloud API can be found here. Here is our complete Python CBT-BB conversion script; # Please visit http://selenium-python.readthedocs.io/ for detailed installation and instructions # Getting started: http://docs.seleniumhq.org/docs/03_webdriver.jsp # API details: https://github.com/SeleniumHQ/selenium#selenium # Requests is the easiest way to make RESTful API calls in Python. You can install it by following the instructions here: # http://docs.python-requests.org/en/master/user/install/ import unittest from selenium import webdriver import requests import os import httpx class BasicTest(unittest.TestCase): def setUp(self): #get rid of the old way of doing auth with just an API key self.apiKey = '' self.api_session = requests.Session() self.test_result = None self.screenshot_dir = os.getcwd() + '/screenshots' self.screenshotName1 = 'SS1.png' self.deviceRunID = "" self.projectID = "" self.RunId = "" #old platformName has been split into platformName and osVersion capabilities = { 'bitbar_apiKey': '', 'platform': 'Linux', 'osVersion': '18.04', 'browserName': 'firefox', 'version': '101', 'resolution': '2560x1920', } # start the remote browser on our server self.driver = webdriver.Remote( desired_capabilities=capabilities, #the hub is changed, also not sending the user and pass through the hub anymore #US hub url: https://appium-us.bitbar.com/wd/hub command_executor="https://us-west-desktop-hub.bitbar.com/wd/hub" #EU hub url ) self.driver.implicitly_wait(20) def test_CBT(self): # We wrap this all in a try/except so we can set pass/fail at the end try: # load the page url print('Loading Url') self.driver.get('http://crossbrowsertesting.github.io/selenium_example_page.html') # maximize the window - DESKTOPS ONLY #print('Maximizing window') #self.driver.maximize_window() #check the title print('Checking title') self.assertEqual("Selenium Test Example Page", self.driver.title) # take a screenshot and save it locally print("Taking a Screenshot") self.driver.get_screenshot_as_file(self.screenshot_dir + '/' + self.screenshotName1) # change pass to SUCCEEDED self.test_result = 'SUCCEEDED' except AssertionError as e: # delete cbt api calls # change fail to FAILED self.test_result = 'FAILED' raise def tearDown(self): print("Done with session %s" % self.driver.session_id) if self.test_result is not None: #get all necessary IDs of current session response = requests.get('https://us-west-desktop-hub.bitbar.com/sessions/' + self.driver.session_id, auth=(self.apiKey, '')).json() self.deviceRunID = str(response["deviceRunId"]) self.projectID = str(response["projectId"]) self.RunId = str(response["testRunId"]) # Here we make the api call to set the test's score requests.post('https://cloud.bitbar.com/api/v2/me/projects/' + self.projectID + '/runs/' + self.RunId + '/device-sessions/' + self.deviceRunID, params={'state': self.test_result}, auth=(self.apiKey, '')) # let's take our locally saved screenshot and push it back to BitBar! # First we start by declaring the 'params' and 'files' variables to hold our Screenshot name and location. params = { 'name': self.screenshotName1, } files = { 'file': open(self.screenshot_dir + '/' + self.screenshotName1, 'rb'), } # Now we build out our API call to push our locally saved screenshot back to our BitBar Project print("Uploading our Screenshot") response = httpx.post('https://cloud.bitbar.com/api/v2/me/projects/' + self.projectID + '/runs/' + self.RunId + '/device-sessions/' + self.deviceRunID + '/output-file-set/files', params=params, files=files, auth=(self.apiKey, '')) # Here we check that our upload was successfull if response.status_code == 201: print("Screenshot Uploaded Successfully") else: print("Whoops, something went wrong uploading the screenshot.") self.driver.quit() if __name__ == '__main__': unittest.main(warnings='ignore') Thanks for reading along, I hope this helps your conversion to BitBar! Happy Testing!