Command line arguments in Jenkins plugin
We have just replaced our old home-grown Jenkins integration with the new 'official' SmartBear Jenkins-TestComplete integration. I find it very easy to work with and it has made our Jenkins integration more 'flashy' and easy to work with for people who are not working with the test system on a daily basis. One feature of our old setup was not possible to reproduce in the new. We sometimes need to run the test on branches of the code (other than the master branch). In our SUT this can be specified in the URL and in the old integration we called TC/TE with a command line argument to specify the branch (if different from master). Unfortunately the new Jenkins integration does not allow for arguments. Adding a field in the UI where you can specify a command line parameter to TestComplete/TestExecute would be nice. This field of course has to allow for variables, so you can pass the information round. This problem can of course be solved in other ways (e.g. dumping the information in a file or environment variable on the slave in Jenkins and read it in TC/TE), but we find our current implementation the most satisfactory. If someone has suggestions on how to implement an alternative, we will be glad to hear them. Yours, Søren Harder, QA engineer, Zmags
New Jenkins TestComplete Plugin Released to publish MHT reports in JUnit format
Hi, I am Fernando Miguélez and I have been a user of TestComplete and TestExecute for almost a year. We use those both to test mainly Flex applications. We use Jenkins actively to continuously build and deploy software but also to perform integration (unit) and functional (with TestComplete/TestExecute) testing. We have been looking forward to hearing from an official Jenkins plugin so TestComplete/TestExecute tests could be more maneageable. Our daily functional tests are generated in MHT format and it takes a lot of manual intervention to figure out what happened with every functional group (we have to download it and open int in Internet Explorer that we happen to use only for that, Firefox or Chrome are our preferred browsers). After waiting for so long and after gathering some experience with Jenkins development I decided to create my own Jenkins plugin. Today I would like to proudly announce the availability of new Jenkins Plugin: TestComplete xUnit Plugin. Just look for it in "Manage Plugins" option of your Jenkins installation. This plugin basically converts MHT TestComplete/TestExecute reports into JUnit format that Jenkins can interpret and handle, so you can benefit from quick failure list review, tendency graphs, failure history, etc. I am not affiliated with Smartbear and my release happened to coincide in time with official release of official TestComplete Jenkins Plugin. My plugin is lighter and only applies to publishing. Official Jenkins Plugin from Smartbear is a full blown plugin to directly invoke TC/TE from Jenkins but leaves small room for customizations. In our case we already had set up our integration with Jenkins by means of many tweaks (batch scripts, Jenkins slaves setup, execution rules, etc.) so we only needed the reporting part. Thus if you start out with a fresh Jenkins integration you may prefer to go with official plugin, but if you happen to come from an existing integration my plugin could be more helpful. I hope you enjoy it. ;-)
TestComplete Jenkins Enhancement
In the current version of the Jenkins plugin for TestComplete there are fields for user and password if running test as an interactive user session. Today these are freeform. I would like to make the recommendation that the plugin use the built in Jenkins credentials manager to select the user to run as from. In today's world with the need for increased security, we are required to have passwords rotate regularly. If we have hundreds of test jobs built out in Jenkins, we would have to modify each and every one of these each time passwords rotate. If the credentials manager is used, you can go to one location, and update the password for the account and not have to worry about modifying each individual job. Thoughts? Anyone else have this issue? Have you found other ways to address?
TestEngine + Jenkins Plugin: Execute list of Project/Suite/Case SoapUI tests
In their current implementation 1.) the TestEngine needs to be provided a zip with all required files by the calling client 2.) the TestEngine Jenkins plugin will execute one Project/Suite/Case per Jenkins job Problem: This setup does not scale to execute many Project/Suite/Case SoapUI tests in a row Suggestion: ad 1.) a.) Provide TestEngine the ability to access/checkout GIT repos containing SoapUI Projects b.) Add an API to TestEngine that will take the Project/Suite/Case/... parameters and pick all required files from above local checkout area --> Like this no files need to be created/transfered to the TestEngine server What still needs to be done is the specification of the list of Project/Suite/Case to be executed. This could be provided by one or more config files maintained in the GIT repo(s) ad 2.) In case the TestEngine runs on the Jenkins slave(s), the Jenkins job could - checkout the GIT repo(s) - locate the config file(s) containing the list of /Project/Suite/Case SoapUI Tests in the checkout area (e.g. via parameter) - loop the config file(s) calling the TestEngine API with Project/Suite/Case/... parameters --> like this - The TestEngine is running as a server (contrary to testruner.sh) - The invocation of many Project/Suite/Case's will run with minimal overhead (presumably msecs instead of secs) - all required files are taken from a repo -> any updates propagate to TestEngine automatically Remark: This type of automated execution of SoapUI Test from different Project/Suite/Case needs to complemented with a suitable test report (e.g. csv file with Project/Suite/Case/Step, timeStamp, Status, ExecTime, ErrorMsg, ...
Jenkins TestComplete jobs should clean up processes
Usually a Jenkins build job injects cookies environment variables to processes it starts ("JENKINS_SERVER_COOKIE" and "HUDSON_SERVER_COOKIE"). Which are inherited into all child processes. And after job is finished, those processes are killed if still are left running. This does not work with TestComplete tests. My guess is, that this is because the TestComplete windows service is already running when job starts. Somehow TestComplete/TestExecute and all processes which are started by the scripts (e.g.via Win32API.WinExec) needs these cookie injected. I hope the Jenkins plugin API provides the means to retrieve the current cookie value.in Jenkins Plugin, mark tests as skipped
I would like to have the ability to mark tests as "skipped" (yellow in result graph) in JUnit . My assumption was, that tests resulting in log level "warning" are marked as such, but these are marked as "passed". An option to mark "warning" test results (MHT) as "skipped" (in JUnit-like test result) would be sufficient.
