TestComplete with Zephyr
In this post, we are going to talk about SmartBear’s UI testing tool, TestComplete, and writing the results to Zephyr in an automated fashion. When we think about using TestComplete with any test management tool, it can really be accomplished in two ways: Natively inside TestComplete or integrating with some CI-CD system. When we are using Zephyr, both ways will utilize the Zephyr REST API. When we link to Zephyr natively from TestComplete, it is a script heavy approach. An example of that can be found here. Today we are going to go into detail about using TestComplete, with a CI-CD system, and sending the results to Zephyr. Now let’s talk about automating this process. The most common way to automate triggering TestComplete tests is through one if its many integrations. TestComplete can integrate to any CI-CD system as it has Command Line options, REST API options, and many native integrations like Azure Dev Ops or Jenkins. The use of a CI-CD system makes managing the executions at scale much easier. The general approach to this workflow would be a two-stage pipeline something like: node { stage('Run UI Tests') { // Run the UI Tests using TestComplete stage('Pass Results') { //Pass Results to Zephyr } } First, we trigger TestComplete to execute our tests somewhere. This could be a local machine or a cloud computer, anywhere, and we store the test results in a relative location. Next, we use a batch file (or alike) to take the results from that relative location, send them to Zephyr. When executing TestComplete tests, there are easy ways to write the results to a specific location in an automated fashion. We will look at options through the CLI as well as what some of the native integrations offer. Starting with the TestComplete CLI, the /ExportSummary:File_Name flag will generate a summary report for the test runs, and save it to a fully qualified or relative path in Junit-XML structure. At a basic level we need this: TestComplete.exe <ProjectSuite Location> [optional-arguments] So something like this: TestComplete.exe "C:\Work\My Projects\MySuite.pjs" /r /ExportSummary:C:\Reports\Results.xml /e The /ExportSummary flag can be stored in a relative or fully qualified directory. We could also use one of TestComplete’s many native integrations, like Jenkins and specify in the settings where to output results: Now that our TestComplete tests are executing, and the results are writing to a relative location we are ready for stage 2 of the pipeline, sending the results to Zephyr. So now let’s send our results to Zephyr. I think the easiest option is to use the Zephyr API, and the Auto-Create Test Case option to true. The command below is a replica of what you would use in a batch file script in the pipeline. curl -H "Authorization: Bearer Zephyr-Token-Here" -F file= Relative-Location-of-Report-Here\report.xml;type=application/xml "https://api.zephyrscale.smartbear.com/v2/automations/executions/junit?projectKey=Project-Key-Here&autoCreateTestCases=true" After you modify the API token, relative location, and project key you are good to run the pipeline. The pipeline should look something like this: After we run the pipeline let’s jump into Jira to find to confirm the results are populating. Even with execution data: Also, with transactional data to analyze the failed test steps:
TestComplete URL for the Jenkins TestComplete report page
I want to offer a way to-get-URL-of-TestComplete-Test-Results-of-a-Jenkins-Build without accessing the Jenkins controller. Outside of those with an admin role for the Jenkins instance, granting teams access to the Jenkins controller is a security risk. TestComplete uses an epoch timestamp as part of the report page URL. This makes it a challenge to use the URL unless you know how to get the timestamp. Fortunately there is an easy way to get it. Once the test plan completes, the results become available using the Jenkins-TestComplete API. Any Jenkins ID that has read access to view the job will have access to use the API. In your job, once the test is completed, use the API to parse the xml for the URL. This allows you to keep your Jenkins instance safe and obtain the report URL for use in your job (such var passed to outgoing email). Here is an example of using a windows batch build step in a freestyle job, after the TestComplete build step. Echo off curl -X GET "http://jenkins:8080/job/folder/job/name/%BUILD_NUMBER%/TestComplete/api/xml" --user %ID%:%pwd%>Test_Results.txt powershell -c "((((gc Test_Results.txt) -replace '<url>','@') -replace '</report>','@') | ForEach-Object { $_.split('@') } | Select-String -Pattern '</url>' -SimpleMatch ) -replace '</url>','' | set-content TestComplete_URL.txt" if exist TestComplete_URL.txt set /p TestComplete_URL=<TestComplete_URL.txt Once you have the URL, you can use the inject plugin to use it in other steps of your job. For example, you can create a var to be used in an outgoing email. echo Line_1=$JOB_BASE_NAME ^<a href^=^"%TestComplete_URL%^"^>test results^</a^>: >>email.properties Hope this information is of use.
TestComplete BDD Integration - Need Your Feedback
Hello TestComplete Customers, Thank you for making contributions to SmartBear community. I am Product Manager for TestComplete. In the upcoming releases, we are looking to implement TestComplete BDD integration. In case this is something interesting to you, please reach out to [email protected]. I would like to hear your use case and get your thoughts on how we should go about implementing this in TestComplete. Look forward to speaking with you. Best, NikhilTest Execution Summary Not Visible on Jenkins
I’m no longer able to view the Test Execution Summary on Jenkins. My test suite completes successfully, and I can still see the brief “failed test” overview under the Tests tab. However, the usual detailed summary within the TestComplete Test Results view is no longer visible. When I try to open the log from that view, the page loads but remains blank. This issue began occurring recently. My current workaround is to download the log file from the TestComplete Test Results tab, then upload and view it manually in TestComplete. Since I review multiple different test suites daily, this workaround is becoming inefficient. any advice? thanks in advance!Checking API Status in TestComplete
Introduction I first saw the need to verify the state of an API several years ago with an application that used an address validation API in several of it's operations. If this API was down or did not respond in a timely manor, many of the automated test cases would run and fail. In this case, and in others, I have found that doing a simple call to the API to check the returned status code allowed me to skip, fail or log a warning with a logical message instead of allowing the application to fail with another less direct error message due to the API issue. The aqHttp Object The TestComplete aqHttp object and it's methods are very useful for performing simple checks like this and are also useful for other more complex tasks like leveraging an API to return a test data set or even verifying a certain data is returned prior to running tests against the UI that depend on the data. Sending and Receiving HTTP Requests From Script Tests More Complete API Testing Most proper API testing should be done using a tools like ReadyAPI or SoapUI. Both of these tools will integrate with TestComplete or can be used alone and will provide much more capabilities and automation options. Integration With ReadyAPI and SoapUI Code Example Here I have provided a working example of how to code a Get request using 'aqHttp.CreateRequest' to confirm an API returns a status code of 200 and it will log the returned records. function sendGetRequest() { let resourcePath ="https://restcountries.com/v3.1/all" let resourceQuery = "?fields=name,capital"; let url = resourcePath + resourceQuery; try { // Send GET request let response = aqHttp.CreateRequest("GET", url, false).Send(); // Check for successful response if (response.StatusCode === 200) { // Parse JSON response let allData = JSON.parse(response.Text); Log.Message("Total records received: " + allData.length); // Process each record allData.forEach((record, index) => { Log.Message("Record " + (index + 1) + ": " + JSON.stringify(record)); }); return true; // Send a bool back to the calling function. } else { throw new Error("Failed to fetch data. Status code: " + response.StatusCode); } } catch (error) { Log.Error("Error during request or data processing: " + error.message); } } Enhancements You could accept parameters for the resourcePath and resourceQuery. Parameterize the logging loop run or remove it. Return the JSON to the calling function for use. Perform other tasks based on the return code. Conclusion With the growing use of API calls in desktop applications and the fact that APIs are almost the foundation of any web site checking an API before a test case run is almost a requirement for consistent test runs and good error logging. This small script can bring big rewards to your test runs and reports. Cheers! I hope you find it as useful as I have! If you find my posts helpful drop me a like! 👍 Leave a comment if you want to contribute or have a better solution or an improvement. 😎 Have a great dayTestEngine external JAR files - error - unable to resolve class
Background - We have test engine hosted on some linux server and we execute tests suing CURL cmd or POSTMAN. Given some external JARs When we add external JARs in /bin/ext folder of test engine server and re-start the test engine And run the tests using curl command from locally or using POSTMAN Then it throws error for groovy script test step saying unable to resolve class <className present in external JARs> on import statement itself. We have also configured .vmoptions of test engine server to read ext JARs but still it throws error. Locally, In ReadyAPI, we are placing those JARs in /bin/ext/ folder and it works perfectly fine. Any solution to this problem?
ReadyAPI changes in background - Git Integration
I'm using ReadyAPI 3.48.0 together with GitLab. After pulling the latest version of the project from GitLab and not making any changes at all, I can see that ReadyAPI makes hundreds of changes in the background. Adding the following under con:settings: xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance The next team member that pushes automatically has the same thing removed in the same files. Is there a way to prevent this from happening?gcloud authentification with service account
Hi community, we are trying to check our google storage buckets using ready api. For this we need a token to access the gcs. It works on my local machine inserting my personal token directly in the oauth token field but, as you might understand, I don't want this to be stored in any gitlab pipeline or log file. Has any body already gathered experience with connection rap to the google storage, and has come across a solution for automatic token generation?
Is it possible to integrate the Azure TestPlan Test cases and the ReadyAPI tests ?
I have created manual test cases in Azure devops Test Plan. I have automated the manual test case functionality using ReadyAPI. Now i want to link the Azure Test Plan test cases while running the ReadyAPI tests via the Azure pieplines. Is it possible to do so ? I am using the Azure devops extension named "ReadyAPI Test for Azure Devops" and "Publish Test Results" in my pipeline.
