Test Execution Summary Not Visible on Jenkins
I’m no longer able to view the Test Execution Summary on Jenkins. My test suite completes successfully, and I can still see the brief “failed test” overview under the Tests tab. However, the usual detailed summary within the TestComplete Test Results view is no longer visible. When I try to open the log from that view, the page loads but remains blank. This issue began occurring recently. My current workaround is to download the log file from the TestComplete Test Results tab, then upload and view it manually in TestComplete. Since I review multiple different test suites daily, this workaround is becoming inefficient. any advice? thanks in advance!
TestComplete with Zephyr
In this post, we are going to talk about SmartBear’s UI testing tool, TestComplete, and writing the results to Zephyr in an automated fashion. When we think about using TestComplete with any test management tool, it can really be accomplished in two ways: Natively inside TestComplete or integrating with some CI-CD system. When we are using Zephyr, both ways will utilize the Zephyr REST API. When we link to Zephyr natively from TestComplete, it is a script heavy approach. An example of that can be found here. Today we are going to go into detail about using TestComplete, with a CI-CD system, and sending the results to Zephyr. Now let’s talk about automating this process. The most common way to automate triggering TestComplete tests is through one if its many integrations. TestComplete can integrate to any CI-CD system as it has Command Line options, REST API options, and many native integrations like Azure Dev Ops or Jenkins. The use of a CI-CD system makes managing the executions at scale much easier. The general approach to this workflow would be a two-stage pipeline something like: node { stage('Run UI Tests') { // Run the UI Tests using TestComplete stage('Pass Results') { //Pass Results to Zephyr } } First, we trigger TestComplete to execute our tests somewhere. This could be a local machine or a cloud computer, anywhere, and we store the test results in a relative location. Next, we use a batch file (or alike) to take the results from that relative location, send them to Zephyr. When executing TestComplete tests, there are easy ways to write the results to a specific location in an automated fashion. We will look at options through the CLI as well as what some of the native integrations offer. Starting with the TestComplete CLI, the /ExportSummary:File_Name flag will generate a summary report for the test runs, and save it to a fully qualified or relative path in Junit-XML structure. At a basic level we need this: TestComplete.exe <ProjectSuite Location> [optional-arguments] So something like this: TestComplete.exe "C:\Work\My Projects\MySuite.pjs" /r /ExportSummary:C:\Reports\Results.xml /e The /ExportSummary flag can be stored in a relative or fully qualified directory. We could also use one of TestComplete’s many native integrations, like Jenkins and specify in the settings where to output results: Now that our TestComplete tests are executing, and the results are writing to a relative location we are ready for stage 2 of the pipeline, sending the results to Zephyr. So now let’s send our results to Zephyr. I think the easiest option is to use the Zephyr API, and the Auto-Create Test Case option to true. The command below is a replica of what you would use in a batch file script in the pipeline. curl -H "Authorization: Bearer Zephyr-Token-Here" -F file= Relative-Location-of-Report-Here\report.xml;type=application/xml "https://api.zephyrscale.smartbear.com/v2/automations/executions/junit?projectKey=Project-Key-Here&autoCreateTestCases=true" After you modify the API token, relative location, and project key you are good to run the pipeline. The pipeline should look something like this: After we run the pipeline let’s jump into Jira to find to confirm the results are populating. Even with execution data: Also, with transactional data to analyze the failed test steps:
Checking API Status in TestComplete
Introduction I first saw the need to verify the state of an API several years ago with an application that used an address validation API in several of it's operations. If this API was down or did not respond in a timely manor, many of the automated test cases would run and fail. In this case, and in others, I have found that doing a simple call to the API to check the returned status code allowed me to skip, fail or log a warning with a logical message instead of allowing the application to fail with another less direct error message due to the API issue. The aqHttp Object The TestComplete aqHttp object and it's methods are very useful for performing simple checks like this and are also useful for other more complex tasks like leveraging an API to return a test data set or even verifying a certain data is returned prior to running tests against the UI that depend on the data. Sending and Receiving HTTP Requests From Script Tests More Complete API Testing Most proper API testing should be done using a tools like ReadyAPI or SoapUI. Both of these tools will integrate with TestComplete or can be used alone and will provide much more capabilities and automation options. Integration With ReadyAPI and SoapUI Code Example Here I have provided a working example of how to code a Get request using 'aqHttp.CreateRequest' to confirm an API returns a status code of 200 and it will log the returned records. function sendGetRequest() { let resourcePath ="https://restcountries.com/v3.1/all" let resourceQuery = "?fields=name,capital"; let url = resourcePath + resourceQuery; try { // Send GET request let response = aqHttp.CreateRequest("GET", url, false).Send(); // Check for successful response if (response.StatusCode === 200) { // Parse JSON response let allData = JSON.parse(response.Text); Log.Message("Total records received: " + allData.length); // Process each record allData.forEach((record, index) => { Log.Message("Record " + (index + 1) + ": " + JSON.stringify(record)); }); return true; // Send a bool back to the calling function. } else { throw new Error("Failed to fetch data. Status code: " + response.StatusCode); } } catch (error) { Log.Error("Error during request or data processing: " + error.message); } } Enhancements You could accept parameters for the resourcePath and resourceQuery. Parameterize the logging loop run or remove it. Return the JSON to the calling function for use. Perform other tasks based on the return code. Conclusion With the growing use of API calls in desktop applications and the fact that APIs are almost the foundation of any web site checking an API before a test case run is almost a requirement for consistent test runs and good error logging. This small script can bring big rewards to your test runs and reports. Cheers! I hope you find it as useful as I have! If you find my posts helpful drop me a like! 👍 Leave a comment if you want to contribute or have a better solution or an improvement. 😎 Have a great dayTestComplete Allure Jenkins Integration
Did any one tried TestComplete Allure Jenkins Integration. I am unable to see Test steps when using /exportsummary.Its working fine but showing basic detail and when using /exportlog then its unable to parse xml file. I want to see TestLog also in Allure html file.
Use Variables in the sql connectionstring
Hi there, I'm using 2 environments and 2 databases: 1 to create the testcases and 1 to execute them I want to check the results with DBTables Custom queries (SSMS) and I have i.e. next connectionstring: How CAN I make this string Variable ? When I login at the ALPHA-environment I want in the string ALPHA and server01 When I login at the BETA-environment I want in the string BETA and server02 Greetings, Sjef van Irsel
VS 2019 - Failed to initialize the TFS API Library
When we try to access the Server Path and add a Server location to access the Online Repo getting an error as Steps to reproduce: 1) Navigate to Tools -> Options -> Source Control and select 'Current Source Control Plugin ' as 'Microsoft Visual Studio 2019' [already installed locally and Source path / Workspace is set] 2) Navigate to File -> Source Control -> Open From Team Foundation Server... 3) Click on the ... button at the 'Server Path:' section.
