TestExecute and Azure Pipelines: Understanding and upgrading test result format
Hi there,
I have set up a Pipeline as part of a build. The test is executed successfully on the self-hosted agent, which has a TestExecute license and a ".trx" file is produced.
I have noticed that this file is very scarce in content. Regarding the individual tests, I can only gather the outcome and test duration.
See the example below, most of the information is made up of identifiers.
<?xml version="1.0" encoding="utf-8"?>
<TestRun id="f20edbe2-19c8-485f-85fd-c3fd3a76de11" name="XXXXX 2023-05-22 09:07:42" runUser="XXXXXX" xmlns="http://microsoft.com/schemas/VisualStudio/TeamTest/2010">
<Times creation="2023-05-22T09:07:42.3114566+02:00" queuing="2023-05-22T09:07:42.3114566+02:00" start="2023-05-22T09:07:16.1427079+02:00" finish="2023-05-22T09:07:46.3304395+02:00" />
<TestSettings name="default" id="bcc38cab-07e3-4179-9708-f8d4e64eb4f4">
<Deployment runDeploymentRoot="buildserver_XXX_2023-05-22_09_07_42" />
</TestSettings>
<Results>
<UnitTestResult executionId="da6d90e9-e78d-4dfc-bc35-6c0423463656" testId="5b32f886-8953-8e58-5cce-00235d5daff7" testName="Launch application" computerName="XXX" duration="00:00:18.1977103" startTime="2023-05-22T09:07:22.7142290+02:00" endTime="2023-05-22T09:07:40.9119393+02:00" testType="13cdc9d9-ddb5-4fa4-a97d-d965ccfc6d4b" outcome="Passed" testListId="8c84fa94-04c1-424b-9868-57a2d4851a1d" relativeResultsDirectory="da6d90e9-e78d-4dfc-bc35-6c0423463656" />
</Results>
<TestDefinitions>
<UnitTest name="Launch application" storage="XXXX" id="5b32f886-8953-8e58-5cce-00235d5daff7">
<Execution id="da6d90e9-e78d-4dfc-bc35-6c0423463656" />
<TestMethod codeBase="XXX\Suite.pjs" adapterTypeName="executor://tctestexecutor/v2" className="AppSuite" name="XXX" />
</UnitTest>
</TestDefinitions>
<TestEntries>
<TestEntry testId="5b32f886-8953-8e58-5cce-00235d5daff7" executionId="da6d90e9-e78d-4dfc-bc35-6c0423463656" testListId="8c84fa94-04c1-424b-9868-57a2d4851a1d" />
</TestEntries>
<TestLists>
<TestList name="Results Not in a List" id="8c84fa94-04c1-424b-9868-57a2d4851a1d" />
<TestList name="All Loaded Results" id="19431567-8539-422a-85d7-44ee4e166bda" />
</TestLists>
<ResultSummary outcome="Completed">
<Counters total="1" executed="1" passed="1" failed="0" error="0" timeout="0" aborted="0" inconclusive="0" passedButRunAborted="0" notRunnable="0" notExecuted="0" disconnected="0" warning="0" completed="0" inProgress="0" pending="0" />
</ResultSummary>
</TestRun>
I have some questions regarding this file and in general what can I expect from testExecute:
1) First, where can I find a format guide for `.trx` files?
2) Can I customize it to include custom metrics? I'm pondering if this a suitable format for aggregating a database of test results
3) Is it possible to obtain a more full-fledged result format as with TestComplete logs? So a html zip with screenshots, expected and actual results, etc
Thank you!
Hi Zgibanez,
I had the same issue for logging as you do until I found the setting below.
- When editing a pipline, go the the TC test adapter installer.
- Search for the "Save logs for" option.
- Luukdb_0-1685440909157.png
Select the desired option
- Luukdb_1-1685440944312.png
Click on the i icon
- Luukdb_2-1685440982543.png
Link the TestComplete log to the test result
- Luukdb_3-1685441017985.png
After you run the pipeline you will get the logs just as in TestComplete itself. To find the logs open the pipeline who is finished and select the "Tests" tab.
Luukdb_4-1685441126788.png
Your screen will look like this.
Luukdb_5-1685441150109.png
Small tip: select the aborted filter en select the passed filter. This way you will also see the logs of passed tests.
Luukdb_6-1685441213362.png
Select the testrun below, and in the attachments tab you will find the logs, and you will be able to download the logs.
Luukdb_7-1685441300327.png
And your logs will look like the one from TC itself
Luukdb_8-1685441335758.png
I am not sure if you are able to customize this outcome but this is way more detailed then the log created by Azure itself.
I hope this is what you are looking for!
