Getting one log entry for TFS / Azure devops Tests
Hi everyone,
I use TFS build (same thing than Azure DevOps pipeline) to launch a TFS test suite via TestComplete.
Everything works fine, but TC create a new log entry for each test case. It's possible to create a unique log entry with sub-log entry like TC does when launching the Execution Plan directly from it ?
OK, before I begin: this only works if you run your VS Test task in Azure using the Test Assemblies config and NOT from Test Plan. I believe this is due to the fact that Azure runs each test in a test plan individually, in its own order, and not as part of a grouped test set that TC controls – this is another reason you cannot force the pipeline to stop from an Event Controller.
If you attempt to run from Test Plans using this approach, you will only receive the log for the last test executed before the OnStopTest() event attempts to save the project logs. Even if you exclude Event Controllers and attempt to use Log.SaveLogAs() in your last test case instead- you’ll get the same results, only details from the last test completed.
One thing I did not test was the command line approach mentioned in my second comment (reference). This may provide the same result as the workaround I describe below, but essentially adding the line below in your TC test adapter pipeline task (see more info here😞
/ExportSummary:File_Name
The method I originally suggested will export the entire test log, although it creates an mht file for each test and not 1 for all tests. You still need to use Log.SaveResultsAs on the last test item only (e.g. OnStopTest, see reference). Just use the correct path to your new MHT_Files folder within the specified project. Take note of the passed parameters: Log.SaveResultsAs(fName, lsMHT, true, 0).
function EventControls_OnStopTest(Sender)
{
var foundFiles, aFile;
var DateTime = aqConvert.DateTimeToFormatStr(aqDateTime.Today(), aqString.Replace("%m/%d/%y", "/", "-"));
var fName = Project.Path + "MHT_Files\\Summary-" + DateTime + ".mht";
foundFiles = aqFileSystem.FindFiles(Project.Path + "MHT_Files\\", "*.mht");
Log.SaveResultsAs(fName, lsMHT, true, 0);
if (foundFiles != null) {
Log.Checkpoint("Summary Report Saved - " + Project.Path + "MHT_Files\\");
}
else {
Log.Message("No files were found.");
}
}
Here's an example Test Execution Plan and EventControler in TestComplete:
You would then create a new "Publish Artifact" task in your pipeline to upload the file to the results.
** use $(System.DefaultWorkingDirectory)\<ProjectSuite Name>\<Project Name>\MHT_Files
Insure the rest of your pipeline settings are correct:
And voilà - now you should get a 1 file summary report attached to your pipeline run, which can be found and configured here in Azure:
Sadly, this is not as straight forward as it should be and something I hope SmartBear fixes in the future. I have a workaround for this, but it will take some time to put together the details and screenshots. The main issue I have found is with the Log.SaveResultAs() functionality not actually working as intended (see post here). Details pending...
Otherwise, with the Event handler and on stop event, can I just create my own custom log ? (getting the log item, parsing it then append data in a xml file or whatever).