Hi, Lee_M
thanks for interesting question.
TestComplete has two kind of results representation: ordinary log (which is more technical and mostly need to understand the cause of test failure) and summary (it contains high-level info about pass/failed scenarios).
If you need to show some high-level report for your boss, I'd recommend to use Summary representation (and it can be exported as JUnit and exported in some test reporting system, or just printed as PDF).
Also the next step can be to put the test set in some test orchestration system: Zephyr, AzureDevOps, or Jenkins (which is free to use).
It can give you merged report, scalability, different kind of representations and historical data.
Here some screenshots from Jenkins:
It stores also the "technical" logs, so, you can always can see them and it can options to clean the storage by scheduler.
Another thing which I like in Summary, that you can add all test cases / scenarios which you discussed with you boss or dev. And automate it one by one, and before non-automated-yet test cases just put Runner.Stop() - all the test cases which are not automated will be in Unexecuted group.
So, you boss can see in report the progress - how many tests already automated and what is in backlog. The technical details is usually not so important for management.
Alex,
Thank you very much for your answer
Unfortunately, the best fit solution for me, as you mentioned would be the Jenkins server which is currently out of scope for the next couple of months
A summary (high level) report is exactly what I need to generate, and have found more information on this page but is only available with test plans and not single runs
The JUnit style report is a good option but requires another piece of software which is not always the best option (looking for a link type like Jenkins - eg. - for test results click here - which could be done exporting the html files)
(Does SmartBear recommend any reporting apps ?)
when you export the html files, do you know if there is a way to add the test title to the report (with editing it manually)
-----------------
Not directly related:
Is there a way we can output the text name automatically - that I can add to, say, the event 'onStopTest'
e.g.
Log.Message("you have just ran: " + test.name)
>but is only available with test plans and not single runs
try to use aqTestCase object, maybe it helps
>Does SmartBear recommend any reporting apps
TestComplete has special support of Jenkins, AzureDevOps, Zephyr - these tools are for orchestration, but have their reporting as well.
This tools have their pros and cons. And maybe you need to choose one of it mostly not as reporting tool, but if you have plans to get additional value to your QA processes.
If we talk only about reporting systems, internally, we test how TestComplete works with Allure. And it works 🙂
>when you export the html files, do you know if there is a way to add the test title to the report (with editing it manually)
it depends how do you export the log. If you do it automatically, yes, you can set the name of the file:
TestComplete.exe "C:\My Projects\MySuite.pjs" /run /p:MyProj /exportLog:"C:\TestLogs\Log.html"
>Is there a way we can output the text name automatically - that I can add to, say, the event 'onStopTest'
not sure how you split the code by tests and what do you run, but maybe this helps for you:
function EventControl1_OnLogError(Sender, LogParams)
{
var strFoo = aqConvert.VarToStr(arguments.callee.caller);
Log.Message(strFoo.split(" ")[1].split("(")[0])
}
AGubarev :
> we test how TestComplete works with Allure. And it works
Sounds really promising. Is it possible to prepare another demo or a sort of a HowTo article with an example?
yes, we have a lot of internal things which we will be glad to share.
Thanks sonya_m and TanyaYatskovska we've had a chance to "say" something to community 🙂
For now, we have no ready video/tutorial for that. But I found that we already have something in Community: https://community.smartbear.com/t5/TestComplete-Questions/Allure-Reporting-using-TestComplete-14-1-Junit-or-JavaClasses/m-p/186731/highlight/true#M65490
arguments.callee.caller
I will further investigate this approach, but last time I ran it, it got one of the subfunctions names (eg Close dialogue, which is called multiple times and is not the parent script run)
