TestComplete and Zephyr Enterprise
Hi everyone, This post in mainly for those of you already using both TestComplete and Zephyr Enterprise, or looking into integrating one with the other via the zbot. I'll try to make this discussion have as little gaps in thought as possible, such that I'm not working off of the assumption that everyone is familiar with how the integration works. To begin with, Zephyr Enterprise (ZE) can trigger TestComplete (TC) tests by triggering command line arguments using the zbot (the secure job agent and results parser). The zbot sits on the target location, and when commanded (by ZE) will find the described invocation script path, and use the directed batch file or shell script to launch the automated tests (in our case, the TC tests). It then waits for the execution completion, and looks into another predefined directory for the results file generated by these TC tests in a junit xml format. Attached is what the configuration looks like from the ZE side: Within the dropdown panel that describes the automation framework (where we have selected TestComplete) what this really describes is the junit xml file parser template that the zbot will use to go through our results file to upload our automated TC test results back to ZE. (https://zephyrdocs.atlassian.net/wiki/spaces/ZE/pages/1558445554/Parser+Templates) This means that the zbot will look for certain tags and elements within our junit xml file generated by TestComplete to upload the results back to our test management solution. Typically, a junit xml file generated by TestComplete looks like this: Given the tags, this means that we cannot upload additional information such as attachment files since there is no tags that describe the file location that the supporting docs may reside (i.e the mht files). Now if we wanted to, we could go back in and edit these junit xml files and rerun the tests such that the zbot parses our results (https://zephyrdocs.atlassian.net/wiki/spaces/ZE/pages/1558445158/Vortex+Job+Execution+Events) but that would defeat the purpose of being able to launch our automated TC tests, no? So what we can do is write an additional script, where we have our favorite language of choice run through our junit xml files such that it appends that information for us automatically. This is by no means the right way to do this, just one of the ways! Couple of ways to improve this would be to refactor this script such that it can take additional inputs such as file location (hard coded for now) and also make the attachment subelement recursive so that we can add in multiple attachments. But for now this will do... Now that we have our script that will modify our junit xml results file for us, all we need to do is append a line to our original batch file that triggers our TC job, such that it modifies the generated xml file for us! So the batch file would something like this: where the highlighted portion is just an additional call to the script we wrote above. As you can see the format is familiar for those who have been using command line arguments for TC <TC executable location> <TC pjs. file you want to run> <any additional arugments i.e /r /p /ExportSummary to generate the xml file> <call to our python exe. to run our script> <our python script to append to our xml file> <xml file location ^^> Now our original xml file has been modified to look like this: So now whenever we trigger our TC jobs from within ZE, it will not only carry over the traditional default info such as test case name, run time, failure messages, and execution status, but also the supporting attachment files that we want! More curious users may want to browse our ZE docs to learn how to map it to existing requirements, or add in further information (https://zephyrdocs.atlassian.net/wiki/spaces/ZE/pages/1558445158/Vortex+Job+Execution+Events) On an additional note, we really only went over one portion of the automation integration when it comes to TC and ZE, but we can modify the steps used within this post to apply to the jobs we are running within our CI frameworks as well by adding in an additional build step that will perform the same type of junit xml results file modification we see here. -------------------- How is everyone else utilizing TC with ZE? What additional information do you think could be useful in terms of test management when it comes to your automation jobs? Having supporting attachments, and maybe even automatic code/test coverage is something that can be created with this method, but I'm curious to see what you think could be helpful. Anyhow, I hope everyone is staying safe and sane!2.3KViews5likes0CommentsAzure Pipeline - Visual Studio Test - Test execute fails to run (but was working yesterday)
Well, another day another question.😄 Yesterday my build was working fine and my tests were passing on a given project. Today it seems like myVisual StudioTest suddenly fails with the following message: "C:\Program Files\Microsoft Visual Studio\2022\Enterprise\Common7\IDE\Extensions\TestPlatform\vstest.console.exe" D:\agent\_work\24\s\Src\XXX.UITests_TC\XXX.UITests_TC.pjs /logger:trx "/TestAdapterPath:\"D:\agent\_work\24\s\"" Microsoft (R) Test Execution Command Line Tool Version 17.6.0 (x64) Copyright (c) Microsoft Corporation. All rights reserved. Starting test execution, please wait... A total of 1 test files matched the specified pattern. Failed to get a list of tests from the "D:\agent\_work\24\s\Src\XXX.UITests_TC\XXX.UITests_TC.pjs" file due to the following error: Cannot start process because a file name has not been provided. No test is available in D:\agent\_work\24\s\Src\XXX.UITests_TC\XXX.UITests_TC.pjs. Make sure that test discoverer & executors are registered and platform & framework version settings are appropriate and try again. No changes were made to my project and no change to the pipeline itself. It is a bit strange. I tried running testexecute directly on the machine in question and it is running fine. I was wondering if anyone got this issue and could point me in the right direction. I find the logs to be not very verbose, i don't even see how TestExecute is called... 😕2KViews3likes15Comments[TechCorner Challenge #1] Passing custom command-line arguments as test parameter
Hi Community! This resource we are creating together has gained so much value over the years. Tons of helpful content can be found in the TestComplete Community thanks to all of you! I propose that we take it to the next level by adding even more useful things to the TechCorner tag - a tag where we keep your tech articles, script examples, use cases, etc. Here's your chance to share expertise and contribute. Task:Write a script that will take the value of a custom argument from the TC command line and use this value to run a parameterized test. Difficulty: For example, imagine that you have a web test that takes the name of the browser as a parameter. The script should check the command line TC was launched with, find the /BrowserName parameter, read its value, and pass it to the web test as a parameter. If the command line contains /BrowserName=chrome, the web test should launch Chrome; for /BrowserName=firefox the web test should launch Firefox, etc. Good luck!Solved5.3KViews3likes7CommentsDisable the TextComplete\TestExecute update dialoge automatically (without GUI interaction)
Dear Community, I would like to install TestComplete\TestExecute automatically and then call it via the command line to test certain things. The problem I have is the update dialogue. This appears later and interrupts my test. Dialogues can be suppressed with the /SilentMode parameter, but unfortunately, the following call does not work for me with the latest 15.x.x but worked with latest version of 14.x.x with the following parameters. "C:\Program Files (x86)\SmartBear\TestComplete 15\x64\Bin\TestComplete.exe" "C:\TestCompleteProjects\my_solution.pjs" /run /p:Clone_Me /t: "KeywordTests|Main" /SilentMode /ForceConversion /exit Without the "/SilentMode" parameter, everything works perfectly, but the update dialogue appears later. My questions are: - Can I automatically (without disabling it manually in the settings) deactivate the update check without the /SilentMode parameter? - Is it possible to update automatically without the update dialogue appearing? - Why does my call with the /SilentMode parameter not work? Many thanks in advance. Andreas166Views1like3CommentsError when executing Devops pipeline
I'm trying to run a pipeline that runs a test by test execute, when running the pipeline it opens testExecute but it's not opening the project. Here is the path where my project is: C:\agent_work\1\s\TestComplete\BusinessAutomacao\BusinessAutomacao.pjs Project name:BusinessCSAutomacao Now follow the Pipeline I created: trigger: - master pool: name: TesteAutomatizado steps: - task: VisualStudioTestPlatformInstaller@1 inputs: packageFeedSelector: 'nugetOrg' versionSelector: 'latestPreRelease' - task: InstallTestCompleteAdapter@1 inputs: accessKey: 'keyfake' logsLevel: '0' - task: VSTest@2 inputs: testSelector: 'testAssemblies' testAssemblyVer2: '**\TestComplete\BusinessAutomacao\BusinessAutomacao.pjs' searchFolder: '$(System.DefaultWorkingDirectory)' testFiltercriteria: 'Project=BusinessCSAutomacao' vsTestVersion: 'toolsInstaller' In this example I didn't pass the accessKey I'm using But when running the pipeline, the following error occurs: Does anyone know how to resolve?775Views1like13CommentsTestComplete IntelliJ Plugin still alive somewhere?
Hey folks, we are currently running a TestComplete pilot and we mostly are scripting our tests. We are currently struggling a bit with the auto-completion and in general with the integration of IntelliJ in the workflow with TestComplete. We know there has been a plugin in the past but we are not able to find it anymore. Does someone has experience with it? Also the plugin in VSCode doesn't seem to work great..428Views1like3CommentsInstalling Testexecute on a fresh virtual machine with IB based license silently
Hi, I am looking into migrating to ID based licenses. We are having an environment with temporary virtual machines, som keeping the ability to install and hook up to license silent is key. Have anyone a solution doing this with ID based licensing? The installation part is easy and done. But the setting up of licensing, with only commandline. I have not been able to find anything on this. Thanks 🙂354Views1like1CommentTest execute is not working from batch file, getting microsoft explorer notification F3F8B407 ***
This is my batch file : "C:\Program Files (x86)\SmartBear\TestExecute 15\Bin\TestExecute.exe" "C:\Data\TestExecutioner\KeywordDriven_Framework\KeywordDriven_Framework.pjs" /p:KeywordDriven_Framework /t:"Script|DriverScript|DriverScript" /r /e /SilentMode "PCPEnNextGen_Smoke" when u execute this batch file test execute start and stopped in background.... and not triggering any test cases...467Views1like1CommentRunning tests by tag name in specific order
Hello, I will be running my suite of tests in a CI/CD pipeline (Jenkins) but I realized from the documentation that running tests by tag name (following the CLI commands) does not guarantee the tests will be run in a certain order. (Note:The tests that match the specified tag will be run in an arbitrary order.) While trying to find a workaround to this inconvenience, I realized that it does indeed follow a specific order. The order is given by how the test files are sorted in the file script.tcscript. So, for whoever needs to run their tests using tag names, and also wants to run a "pre-step test" or something like that, all you have to do is open that file on any text editor, and move the row where your test file is, up to the place first place, or at whichever place you want it to be executed. Example: # Assuming all tests have the tag "@Smoke" <folder name="MaiGroup"> <folder name="Group1"> <child name="TestMethod1" key="ZZZZ-YYYYY-VVVVV}" path="File1.js" /> <child name="TestMethod2" key="ZZZZ-YYYYY-VVVVV}" path="File2.js" /> <child name="TestMethod3" key="ZZZZ-YYYYY-VVVVV}" path="File3.js" /> </folder> </folder> # Test run for the @Smoke tag will look like: TestMethod1 TestMethod2 TestMethod3 ----------------------------------------------------------- # Now, if we want to run TestMethod3 first, all we have to do is open the script.tcscript file and move the TestMethod3 to the first place: <folder name="MaiGroup"> <folder name="Group1"> <child name="TestMethod3" key="ZZZZ-YYYYY-VVVVV}" path="File3.js" /> <child name="TestMethod1" key="ZZZZ-YYYYY-VVVVV}" path="File1.js" /> <child name="TestMethod2" key="ZZZZ-YYYYY-VVVVV}" path="File2.js" /> </folder> </folder> # And now, the test run for the @Smoke tag will look like: TestMethod3 TestMethod1 TestMethod2 ----------------------------------------------------------- Hope this helps someone, and maybe the TestComplete team can now update the documentation. Regards.570Views1like0Comments