TestComplete with Zephyr Scale
In this post, we are going to talk about SmartBear’s UI testing tool, TestComplete, and writing the results to Zephyr Scale in an automated fashion. When we think about using TestComplete with any test management tool, it can really be accomplished in two ways: Natively inside TestComplete or integrating with some CI-CD system. When we are using Zephyr Scale, both ways will utilize the Zephyr Scale REST API. When we link to Zephyr Scale natively from TestComplete, it is a script heavy approach. An example of that can be found here. Today we are going to go into detail about using TestComplete, with a CI-CD system, and sending the results to Zephyr Scale. Now let’s talk about automating this process. The most common way to automate triggering TestComplete tests is through one if its many integrations. TestComplete can integrate to any CI-CD system as it has Command Line options, REST API options, and many native integrations like Azure Dev Ops or Jenkins. The use of a CI-CD system makes managing the executions at scale much easier. The general approach to this workflow would be a two-stage pipeline something like: node { stage('Run UI Tests') { // Run the UI Tests using TestComplete stage('Pass Results') { //Pass Results to Zephyr Scale } } First, we trigger TestComplete to execute our tests somewhere. This could be a local machine or a cloud computer, anywhere, and we store the test results in a relative location. Next, we use a batch file (or alike) to take the results from that relative location, send them to Zephyr Scale. When executing TestComplete tests, there are easy ways to write the results to a specific location in an automated fashion. We will look at options through the CLI as well as what some of the native integrations offer. Starting with the TestComplete CLI, the /ExportSummary:File_Name flag will generate a summary report for the test runs, and save it to a fully qualified or relative path in Junit-XML structure. At a basic level we need this: TestComplete.exe <ProjectSuite Location> [optional-arguments] So something like this: TestComplete.exe "C:\Work\My Projects\MySuite.pjs" /r /ExportSummary:C:\Reports\Results.xml /e The /ExportSummary flag can be stored in a relative or fully qualified directory. We could also use one of TestComplete’s many native integrations, like Jenkins and specify in the settings where to output results: Now that our TestComplete tests are executing, and the results are writing to a relative location we are ready for stage 2 of the pipeline, sending the results to Zephyr Scale. So now let’s send our results to Zephyr Scale. I think the easiest option is to use the Zephyr Scale API, and the Auto-Create Test Case option to true. The command below is a replica of what you would use in a batch file script in the pipeline. curl -H "Authorization: Bearer Zephyr-Scale-Token-Here" -F file= Relative-Location-of-Report-Here\report.xml;type=application/xml "https://api.zephyrscale.smartbear.com/v2/automations/executions/junit?projectKey=Project-Key-Here&autoCreateTestCases=true" After you modify the API token, relative location, and project key you are good to run the pipeline. The pipeline should look something like this: After we run the pipeline let’s jump into Jira to find to confirm the results are populating. Even with execution data: Also, with transactional data to analyze the failed test steps:
TestComplete URL for the Jenkins TestComplete report page
I want to offer a wayto-get-URL-of-TestComplete-Test-Results-of-a-Jenkins-Buildwithout accessing the Jenkins controller. Outside of those with an admin role for the Jenkins instance, granting teams access to the Jenkins controller is a security risk. TestComplete uses an epoch timestamp as part of the report page URL. This makes it a challenge to use the URL unless you know how to get the timestamp. Fortunately there is an easy way to get it. Once the test plan completes, the results become available using theJenkins-TestComplete API. Any Jenkins ID that has read access to view the job will have access to use the API. In your job, once the test is completed, use the API to parse the xml for the URL. This allows you to keep your Jenkins instance safe and obtain the report URL for use in your job (such var passed to outgoing email). Here is an example of using a windows batch build step in a freestyle job, after the TestComplete build step. Echo off curl -X GET "http://jenkins:8080/job/folder/job/name/%BUILD_NUMBER%/TestComplete/api/xml" --user %ID%:%pwd%>Test_Results.txt powershell -c "((((gc Test_Results.txt) -replace '<url>','@') -replace '</report>','@') | ForEach-Object { $_.split('@') } | Select-String -Pattern '</url>' -SimpleMatch ) -replace '</url>','' | set-content TestComplete_URL.txt" if exist TestComplete_URL.txt set /p TestComplete_URL=<TestComplete_URL.txt Once you have the URL, you can use the inject plugin to use it in other steps of your job. For example, you can create a var to be used in an outgoing email. echo Line_1=$JOB_BASE_NAME ^<a href^=^"%TestComplete_URL%^"^>test results^</a^>: >>email.properties Hope this information is of use.
