CrossBrowserTesting to BitBar Selenium Script Migration - QuickStart Guide
On June 21, 2022, SmartBear launched web application testing on our unified cloud testing solution that will include both browser and device testing on the BitBar platform! We have listened to our customers and having one product for both web and device testing will better meet your needs. BitBar is a scalable, highly reliable and performant platform with multiple datacenters. On it, you will have access to the latest browsers and devices with additional deployment options to meet your needs, including private cloud and dedicated devices. For more frequently asked questions about the launch of web app testing on BitBar, visit our FAQ. This Quickstart Guide is intended to walk through the conversion of your existing CrossBrowserTesting Selenium tests to use BitBar! We have updated Selenium hubs and API calls that will require conversion, though little else will be required. As with CrossBrowserTesting, we have sample scripts and a Selenium Capabilities Configurator you may use to build out the specific capabilities for the desired tested device. This tool can be found here. To start conversion, you will need your BitBar API key versus the CrossBrowserTesting Authkey. This is a new method to authenticate the user and to make API calls. You may find your BitBar API Key in account settings as described here. Most of the Code examples and talking points for conversion are in reference to the CrossBrowserTesting Selenium Sample script that is available here. All code snippets in this article will be in Python. Now that you have your BitBar API Key, let's alter the original Authkey variable with our new BitBar API key located at line 18 in the CrossBrowserTesting sample script. This step is for connection to the BitBar API for processes such as taking screenshots and setting the status of your tests. # Old CrossBrowser Testing Sample Authkey Variable self.authkey = "<"CrossBrowserTesting Authkey">" # New Bitbar API Key Variable self.apiKey = "<"insert your BitBar API Key here">" In regards to the capabilities used in BitBar, there are a couple things to note. First, we do not need to specify a 'record_video' capability as we do in CrossBrowserTesting. Videos are generated automatically for every test, so we no longer need to provide this capability. Doing so will result in webDriver errors. The second thing to note is that we now also pass the BitBar API Key along with the Capabilities; capabilities = { 'platform': 'Windows', 'osVersion': '11', 'browserName': 'chrome', 'version': '102', 'resolution': '1920x1080', 'bitbar_apiKey': '<insert your BitBar API key here>', } With BitBar we now have four Selenium hub options to choose from. Both US and EU Selenium hubs are available to aid in performance for your location. Separate hubs are also provided depending on the type of device (Desktop vs Mobile) you wish to test against. You may pick the applicable Desktop or Mobile hub closest to your location and replace your existing hub with the updated URL; BitBar Desktop Selenium Hubs; US_WEST:DESKTOP: https://us-west-desktop-hub.bitbar.com/wd/hub EU:DESKTOP: https://eu-desktop-hub.bitbar.com/wd/hub BitBar Mobile Selenium Hubs; US_WEST:MOBILE: https://us-west-mobile-hub.bitbar.com/wd/hub EU:MOBILE: https://eu-mobile-hub.bitbar.com/wd/hub # start the remote browser on our server self.driver = webdriver.Remote desired_capabilities=capabilities command_executor="https://us-west-desktop-hub.bitbar.com/wd/hub" Now that we have our BitBar API Key, and Capabilities and Selenium Hub set up, we can move on to altering our requests for Screenshots and Test Result Status. In the CrossBrowserTesting sample script, we use standalone API requests to create Screenshots. For the BitBar sample scripts, we are doing this with the Selenium driver itself to create the Screenshot and store it locally. Afterwards use the BitBar API to push the locally saved image back to our project. The swagger spec for our BitBar Cloud API can be found here. In line 30 of the BitBar Selenium sample script we set a location to store Screenshots on the local machine. Note, this is set up to store files in a directory called 'Screenshots' in the root folder of your project. self.screenshot_dir = os.getcwd() + '/screenshots' To retrieve a Screenshot and store it, we perform a 'get_screenshot_as_file' call, as seen on line 45 in the BitBar Selenium example script. self.driver.get_screenshot_as_file(self.screenshot_dir + '/' + '1_home_page.png') Now we want to to take our Screenshot and push it back to our project in BitBar. Note that in this case for Python, we are using the 'httpx' module for the API calls back to BitBar. The 'requests' module only supports HTTP 1.1 and we will need a module capable of handling HTTP 2/3 requests. # Let's take our locally saved screenshot and push it back to BitBar! # First we start by declaring the 'params' and 'files' variables to hold our Screenshot name and location. params = { 'name': self.screenshotName1, } files = { 'file': open(self.screenshot_dir + '/' + self.screenshotName1, 'rb'), } # Now we build out our API call to push our locally saved screenshot back to our BitBar Project print("Uploading our Screenshot") response = httpx.post('https://cloud.bitbar.com/api/v2/me/projects/' + self.projectID + '/runs/' + self.RunId + '/device-sessions/' + self.deviceRunID + '/output-file-set/files', params=params, files=files, auth=(self.apiKey, '')) # Here we check that our upload was successfull if response.status_code == 201: print("Screenshot Uploaded Successfully") else: print("Whoops, something went wrong uploading the screenshot.") The final piece of the puzzle is to set our Test Result Status. We have alternate naming conventions for test results, these are 'Succeeded' and 'Failed' for BitBar vs 'Pass' and 'Fail' for CrossBrowserTesting. # CrossBrowserTesting Successful test syntax self.test_result = 'pass' # CrossBrowserTesting Failed test syntax self.test_result = 'fail' # BitBar Successful test syntax self.test_result = 'SUCCEEDED' # BitBar Failed test syntax self.test_result = 'FAILED' Note that in the snippet provided below, we start by performing Get requests for session information. These requests are sent to the same Selenium Hub we are using for the webDriver, so make sure the hub address is set to the same hub used for webDriver. We would recommend to turn this into a variable to avoid having to switch this manually for alternate hubs. These processes are found in the 'tearDown' function of the updated CrossBrowserTesting sample script found here. #get all necessary IDs of current session response = requests.get('https://us-west-desktop-hub.bitbar.com/sessions/' + self.driver.session_id, auth=(self.apiKey, '')).json() deviceRunID = str(response["deviceRunId"]) projectID = str(response["projectId"]) RunId = str(response["testRunId"]) Finally, we set the Test Result with the Post method below using session information retrieved with the Get request above. Note, the URL for the Post request will NOT need to be updated to reflect the specific Selenium hub in use. # Here we make the api call to set the test's score requests.post('https://cloud.bitbar.com/api/v2/me/projects/' + projectID + '/runs/' + RunId + '/device-sessions/' + deviceRunID, params={'state': self.test_result}, auth=(self.apiKey, '')) Now that we have made these changes you are ready to run your test through BitBar! As a summary, we replace our CrossBrowserTesting authKey with the BitBar API Key, set the new Selenium hub address, build new screenshot calls and update the test result function. Quick Reference Documentation; BitBar Web FAQ. Complete documentation with code samples in various languages are found here. Retrieve your BitBar API Key in account settings as described here. BitBar Selenium Capability Configurator and Sample Scripts are found here. CrossBrowserTesting Capability Configurator and Sample Scripts are found here. The Swagger spec for our BitBar Cloud API can be found here. Here is our complete Python CBT-BB conversion script; # Please visit http://selenium-python.readthedocs.io/ for detailed installation and instructions # Getting started: http://docs.seleniumhq.org/docs/03_webdriver.jsp # API details: https://github.com/SeleniumHQ/selenium#selenium # Requests is the easiest way to make RESTful API calls in Python. You can install it by following the instructions here: # http://docs.python-requests.org/en/master/user/install/ import unittest from selenium import webdriver import requests import os import httpx class BasicTest(unittest.TestCase): def setUp(self): #get rid of the old way of doing auth with just an API key self.apiKey = '' self.api_session = requests.Session() self.test_result = None self.screenshot_dir = os.getcwd() + '/screenshots' self.screenshotName1 = 'SS1.png' self.deviceRunID = "" self.projectID = "" self.RunId = "" #old platformName has been split into platformName and osVersion capabilities = { 'bitbar_apiKey': '', 'platform': 'Linux', 'osVersion': '18.04', 'browserName': 'firefox', 'version': '101', 'resolution': '2560x1920', } # start the remote browser on our server self.driver = webdriver.Remote( desired_capabilities=capabilities, #the hub is changed, also not sending the user and pass through the hub anymore #US hub url: https://appium-us.bitbar.com/wd/hub command_executor="https://us-west-desktop-hub.bitbar.com/wd/hub" #EU hub url ) self.driver.implicitly_wait(20) def test_CBT(self): # We wrap this all in a try/except so we can set pass/fail at the end try: # load the page url print('Loading Url') self.driver.get('http://crossbrowsertesting.github.io/selenium_example_page.html') # maximize the window - DESKTOPS ONLY #print('Maximizing window') #self.driver.maximize_window() #check the title print('Checking title') self.assertEqual("Selenium Test Example Page", self.driver.title) # take a screenshot and save it locally print("Taking a Screenshot") self.driver.get_screenshot_as_file(self.screenshot_dir + '/' + self.screenshotName1) # change pass to SUCCEEDED self.test_result = 'SUCCEEDED' except AssertionError as e: # delete cbt api calls # change fail to FAILED self.test_result = 'FAILED' raise def tearDown(self): print("Done with session %s" % self.driver.session_id) if self.test_result is not None: #get all necessary IDs of current session response = requests.get('https://us-west-desktop-hub.bitbar.com/sessions/' + self.driver.session_id, auth=(self.apiKey, '')).json() self.deviceRunID = str(response["deviceRunId"]) self.projectID = str(response["projectId"]) self.RunId = str(response["testRunId"]) # Here we make the api call to set the test's score requests.post('https://cloud.bitbar.com/api/v2/me/projects/' + self.projectID + '/runs/' + self.RunId + '/device-sessions/' + self.deviceRunID, params={'state': self.test_result}, auth=(self.apiKey, '')) # let's take our locally saved screenshot and push it back to BitBar! # First we start by declaring the 'params' and 'files' variables to hold our Screenshot name and location. params = { 'name': self.screenshotName1, } files = { 'file': open(self.screenshot_dir + '/' + self.screenshotName1, 'rb'), } # Now we build out our API call to push our locally saved screenshot back to our BitBar Project print("Uploading our Screenshot") response = httpx.post('https://cloud.bitbar.com/api/v2/me/projects/' + self.projectID + '/runs/' + self.RunId + '/device-sessions/' + self.deviceRunID + '/output-file-set/files', params=params, files=files, auth=(self.apiKey, '')) # Here we check that our upload was successfull if response.status_code == 201: print("Screenshot Uploaded Successfully") else: print("Whoops, something went wrong uploading the screenshot.") self.driver.quit() if __name__ == '__main__': unittest.main(warnings='ignore') Thanks for reading along, I hope this helps your conversion to BitBar! Happy Testing!
Launch Browser in Incognito/Private Mode
Question Thought of sharing the code in the community for launching browsers in their incognito modes. The function is parameterized such a way to run for the browsers Internet Explorer, Edge, Chrome and Firefox. Hope it will be useful for more people. Answer //JScript function runIncognitoMode(browserName){ //var browserName = "firefox" //iexplore,edge,chrome,firefox if (Sys.WaitBrowser(browserName).Exists){ var browser = Sys.Browser(browserName); Log.Enabled = false // To disable the warning that might occur during closing of the browser browser.Close(); Log.Enabled = true // enabling the logs back } if(browserName=="edge"){ Browsers.Item(btEdge).RunOptions = "-inprivate" Delay(3000) Browsers.Item(btEdge).Run(); } else if (browserName=="iexplore"){ Browsers.Item(btIExplorer).RunOptions = "-private" Delay(3000) Browsers.Item(btIExplorer).Run(); } else if (browserName=="chrome"){ Browsers.Item(btChrome).RunOptions = "-incognito" Delay(3000) Browsers.Item(btChrome).Run(); } else if (browserName=="firefox"){ Browsers.Item(btFirefox).RunOptions = "-private" Delay(3000) Browsers.Item(btFirefox).Run(); } Sys.Browser(browserName).BrowserWindow(0).Maximize() }Function to wait for processing to complete
In many cases we have seen that processing times varies so it will wait for the processing to get idle and you can edit the max timeout in this , so this will help in cases of rendering and some process extensive work where we cant use any wait command or any other progress. I tried this most of the CAD Softwares like AutoCAD, SolidWorks, Navisworks, Revit, Aveva PDMS, Bentley Microstation, BricsCAD and it worked well. //function is to Wait for CPU processing to get Idle and with a timeout time function WaitForProcessing() { Log.Message("Memory Usage : " + Sys.Process("PROCESS").MemUsage); var time1 = aqDateTime.Time(); while(Sys.Process("PROCESS").CPUUsage != 0) { Log.CheckPoint("While Loop : cpu usage " + Sys.Process("PROCESS").CPUUsage); aqUtils.Delay(2000,"Wait for Processing"); //Waiting for 2 sec //Timeout for max 10sec var time2 = aqDateTime.Time(); var diff = time2 - time1; if(diff >= 10000) ///Please edit the time accordingly { Log.CheckPoint("Max Timeout for While loop with time : " + diff); break; } } Log.Message("While Loop Completed with Cpu Usage : " + Sys.Process("PROCESS").CPUUsage); }
Get properties of a web page element
Question In this example, we will demonstrate how to get some more information about a web element by using TestComplete. How to find the “Start a topic” button on the community page page and get the following info about it: color, font family, and font size and post the script and the log info below. Answer //JavaScript function test2() { var url = "https://community.smartbear.com/t5/TestComplete-General-Discussions/TechCorner-Challenge-13-Get-properties-of-a-web-page-element/m-p/207539"; Browsers.Item(btChrome).Run(url); var page = Sys.Browser().Page(url); var element = page.FindChildByXPath("//*[@class='NewTopic-link']"); var style = page.contentDocument.defaultView.getComputedStyle(element, ""); Log.Message("The Properties of web page element are as follows") Log.Message("Background Color : " + style.backgroundColor); Log.Message("Font Family : " + style.fontFamily); Log.Message("Font Size : " + style.fontSize); }Code for Tracking Tested App Info On Start Test.
Question I like to know the information about the tested app each test ran on so I wrote up a little code and put it in the OnStartTest Test Engine Event. Answer This will run every time I run a test telling me the tested app info. This is wonderful for tracking one off test runs and which app version a test passed on and which it failed on. https://support.smartbear.com/testcomplete/docs/testing-with/advanced/handling-events/index.html https://support.smartbear.com/testcomplete/docs/reference/events/onstarttest.html?sbsearch=OnStartTe... function EventControl_OnStartTest(Sender) { try { Log.AppendFolder("< EventControl_OnStartTest >"); Log.AppendFolder("Version Information"); var FileName = "C:\\Program Files (x86)\\Some Folder\\TestedApp.exe"; var VerInfo = aqFileSystem.GetFileInfo(FileName).VersionInfo; var FileInf = aqFileSystem.GetFileInfo(FileName); var HostName = Sys.HostName; var dtObj; Log.Message("File Name: " + FileInf.Name); Log.Message("File Version: " + VerInfo.FileMajorVersion + "." + VerInfo.FileMinorVersion + "." + VerInfo.FileBuildVersion + "." + VerInfo.FileRevisionVersion); dtObj = new Date(FileInf.DateLastModified); Log.Message("File Date: " + FileInf.DateLastModified); Log.Message("Host Name: " + HostName); Log.PopLogFolder(); } catch(err) { Process.Halt("Exception: EventControl_OnStartTest - " + err.message); //Stop Test Run. } }TestComplete and Device Cloud Add On
Hi All, Today, I wanted to discuss the Device Cloud add on and its practical usage just a bit further, most of what I'm talking about is pretty well covered in the documentation here: https://support.smartbear.com/testexecute/docs/running/cross-platform-tests/run/command-line.html In short, once we have a stable set of test cases that we'd like to run in parallel across multiple OS and browser configurations, this should be your go-to move! This will make use of a newly released application called "TestExecuteLite.exe", which supports the launching multiple instances of the application - hence the "parallels" as the documentation above refers to. Things to note: TestExecuteLite (henceforth to be called TElite, can launch multiple instances, and is currently supported in the Jenkins plugin/pipeline, Azure DevOps pipeline, and the CMD line) We need to use the "new name mapping" of the Xpath and CSS selectors We should try to make our entire pjs only contains web-app based projects/tests (no LL procedures, desktop based tests/ testedapps, etc.) To start, we need to create a function that will consume our custom command line parameters. The docs above have the same function, but here is mine (which is annotated just a little bit more) config = "" def processCommandLineArguments(): #ParamCount returns an integer of # of parameters used in cmd line args for i in range(0, BuiltIn.ParamCount() + 1): #added 1 to the range above for list indexing to print out the info about the last cmd arg Log.Message(BuiltIn.ParamStr(i)) #print each of the cmd args, looks like for some reason tc cant print the custom arg entry at the very end processCommandLineArgument(BuiltIn.ParamStr(i)) #ParamStr returns a str of each of the cmd line entries ie. /config=Safari def processCommandLineArgument(arg): items = arg.split("=") #using python str split method to create a new list variable with newly split strs as each of the index items if (len(items) != 2): #if the cmd arg didnt use an '=' to divide up (hence not a custom arg) return blank #(since len(items) woudl be 1 if they didnt use '=' in the cmd line args) return #get rid of break characters, replace backslash with nothing for items[0] entries (looking for the custom arg) item = aqString.ToLower(aqString.Replace(aqString.Trim(items[0]), "/", "")) if (item == "config"): #if we find the custom arg that we are looking for (which is "config") Log.Message(items[1]) #this should read as the caps name that we want to run global config # Set the config variable for this unit using the "global" keyword config = items[1] #access the index 1 entry of items (this should be the config name i.e Safari) Now that we can receive the command line parameters dictating which config to use, we need to create another function which will tell TestComplete/TestExecuteLite what those configs are in the JSON format that the Run Remote Browser expects as a part of the parameters. We do it with the code below (once again documented, but I changed it slightly to make it a bit more readable) config_dict = {} #empty dict #implmenting a switch case style dictionary for the get capabilities function instead #(dict support for JSON notation makes it easier to manipulate later) def getCapabilities(argument): #list out some capabilities that we want to use config_dict = { "Safari": {"platform":"Mac OSX 10.15","browserName": "Safari","version": "13","screenResolution": "1366x768","record_video":"true"}, "Edge": {"platform":"Windows 10","browserName":"MicrosoftEdge","version":"79","screenResolution":"1366x768","record_video": "true"}, "Chrome": {"platform":"Windows 10","browserName":"Chrome","version":"80x64","screenResolution":"1366x768","record_video": "true"}, } #get the caps in dictionary format- json-ish standard, otherwise, output "invalid config" capabilities = config_dict.get(argument, "Invalid Config") #assign current testcase name to the caps-dict['name'] capabilities['name'] = str(aqTestCase.CurrentTestCase.Name) + " " + argument + " Test" return capabilities #ouput the cmd line config in caps format - to be consumed in our actual test Now that we have these helper functions, we want to create one more function so that we can generalize our tests to receive the command line arguments from either the cmd line or from our CICD frameworks, such that it kicks off our tests in the correct configurations in our remote browser. def startUp_withConfigs(URL): # Get capabilities processCommandLineArguments() aCap = getCapabilities(config) #get the corresponding capabilities from the getCapabilities() function if (aCap != None): server = "http://hub.crossbrowsertesting.com:80/wd/hub" #defining our CBT server no need to ever change this Browsers.RemoteItem[server, aCap].Run(URL) #launching remote browser Now, if you disable/comment out the first line of your web app functional tests which make references to hard coded local/remote browsers, this startUp_withConfigs(URL) function will take the command line arguments, parse it, and supply the correct capabilities and start up the remote browser session to the specified URL (of course we don't need to make this URL mandatory). If you've build some modular test cases, you can go ahead and start changing/disabling/commenting and replacing the other "blocks" of code to use similarly refactored, generalized code like: def NavigateCurrentBrowser(URL): Browsers.CurrentBrowser.Navigate(URL) def MaximizeCurrentBrowser(): Aliases.browser.BrowserWindow(0).Maximize() to make sure that we continue to use the active remote browser session, stay on the designated web page, and to make sure that our remote browsers are maximized (sometimes elements will face an object not found error if this isn't true, at least on non-mobile browsers). Now your test scripts may look something like this: where the previous lines of code expecting a local browser, or a hard coded remote browser config has been commented out or for your keyword tests, something like the following screenshot attached such that when run through a CICD framework or the CMD line, we achieve as many parallels as we described within the configs section alongside the tests we selected (won't go into deployment configurations for CICD and cmd line since that is also well documented, and without code, so no further annotations needed from me): I'd love to hear if anyone in the community has some more practical usage of the TELite application, or any questions or concerns surrounding the example shown above. Being able to make certain of your web app's functionality across multiple configs of major browsers can certainly help with the traditional browser coverage in testing, but now we are able to do it in parallel using a combination of 3-4 functions. What are some other ways that the community has been using in order to increase your testing velocity when it comes to web app testing?Checking API Status in TestComplete
Introduction I first saw the need to verify the state of an API several years ago with an application that used an address validation API in several of it's operations. If this API was down or did not respond in a timely manor, many of the automated test cases would run and fail. In this case, and in others, I have found that doing a simple call to the API to check the returned status code allowed me to skip, fail or log a warning with a logical message instead of allowing the application to fail with another less direct error message due to the API issue. The aqHttp Object The TestComplete aqHttp object and it's methods are very useful for performing simple checks like this and are also useful for other more complex tasks like leveraging an API to return a test data set or even verifying a certain data is returned prior to running tests against the UI that depend on the data. Sending and Receiving HTTP Requests From Script Tests More Complete API Testing Most proper API testing should be done using a tools like ReadyAPI or SoapUI. Both of these tools will integrate with TestComplete or can be used alone and will provide much more capabilities and automation options. Integration With ReadyAPI and SoapUI Code Example Here I have provided a working example of how to code a Get request using 'aqHttp.CreateRequest' to confirm an API returns a status code of 200 and it will log the returned records. function sendGetRequest() { let resourcePath ="https://restcountries.com/v3.1/all" let resourceQuery = "?fields=name,capital"; let url = resourcePath + resourceQuery; try { // Send GET request let response = aqHttp.CreateRequest("GET", url, false).Send(); // Check for successful response if (response.StatusCode === 200) { // Parse JSON response let allData = JSON.parse(response.Text); Log.Message("Total records received: " + allData.length); // Process each record allData.forEach((record, index) => { Log.Message("Record " + (index + 1) + ": " + JSON.stringify(record)); }); return true; // Send a bool back to the calling function. } else { throw new Error("Failed to fetch data. Status code: " + response.StatusCode); } } catch (error) { Log.Error("Error during request or data processing: " + error.message); } } Enhancements You could accept parameters for the resourcePath and resourceQuery. Parameterize the logging loop run or remove it. Return the JSON to the calling function for use. Perform other tasks based on the return code. Conclusion With the growing use of API calls in desktop applications and the fact that APIs are almost the foundation of any web site checking an API before a test case run is almost a requirement for consistent test runs and good error logging. This small script can bring big rewards to your test runs and reports. Cheers! I hope you find it as useful as I have! If you find my posts helpful drop me a like! 👍 Leave a comment if you want to contribute or have a better solution or an improvement. 😎 Have a great day
