ExportSummary flag only exports the log for one single test case
In our Azure pipeline, I'm using the /ExportSummary flag for Test Execute (version 15.49) to generate a JUnit .xml file with the test results like this:
After all test cases are executed, this log file is successfully written to the disk but it only contains the result for the first executed test case:
After checking the logs, one can see that one log file is generated per executed test case instead of one log file for the whole execution:
When running the same configuration from within TestComplete, one single test exeuction summary for all test cases is created. How can I fix this?
Hi,
> How can I fix this?
No way. (As per my understanding.)
The case is that all CI\CD servers/pipelines use a concept of single, atomic independent (unit-) test. This means that when you command in a pipeline to execute, say, three tests, the runner does not execute them within one single 'session'. Instead, TestExecute is invoked and commanded to execute one test (not necessarily the first one, you should not rely or expect some defined order of execution). When the test ends and TestExecute closes, the runner invokes TestExecute for the second time and commands it to execute another test. And so on and so forth until the list of requested tests is over.
(The same explanation was provided in some other thread here by Support but I don't have it at hand at the moment.)
As a kind of workaround, you may consider to use aqTestCase object provided by TestComplete assuming the wrapping code like this (untested pseudocode):
// @"My Test"
function aTestFactory {
var arTestsList = ['test1()', 'test2()', ...];
for (var aTest in arTestsList) {
var strCurrTest = arTestsList[aTest];
aqTestCase.Begin(strCurrTest);
eval(strCurrTest);
aqTestCase.End();
}
// export log here or use /ExportSummary flag
}
This should provide one test log (because only one "My Test" test has been executed from the runner's point of view) but with the summary for all test cases executed from aTestFactory() function.
Note, however, that aqTestCase is not fully equivalent to test case created in the Execution Plan and there are more or less subtle differences between them.
