Hi,
The order should be:
1. aqPerformance.Start()
2. click button to lauch application
3. wait until application is visible
4. aqPerformance.Check(100)
Should be fine for what you need.
The documentation says it goes down to milliseconds.
https://support.smartbear.com/testcomplete/docs/reference/program-objects/aqperformance/value.html
Although, you have to factor in the time it takes to find you visible object. How much time that will add is hard to say. It depends what you're searching, how you search for it, and possibly how busy the applications object model is.
If it starts any processes or services, it may be slightly quicker to check for those instead? Not sure. You'd need to have a play about with it and see for yourself.
I'll also add (as I do performance testing as well), that unless you are only running this application on specific, controlled, hardware, then such a measurement cannot be accepted as a global given anyway.
Because:
Run it on a high spec i7/SSD based PC ..... launch is super fast.
vs
Run it on an Atom based machine, clogged up with other rubbish installed, and running off a 5400rpm HDD, and the result will be much slower.
Of course, if you control the hardware, then fine.
Just a nitpick:
Application Launch vs Application visible are NOT necessarily the same thing. As Colin_McCrae, there are a number of hardware and environmental factors that could make a determination of when the application actually becomes visible on screen.
Personally, while the features are there, I don't view TestComplete as a tool for that level of performance measurement. It's helpful in keeping track of stuff that has specific requirements, but there's always a variability of error.
Hi Colin,
"Although, you have to factor in the time it takes to find you visible object. How much time that will add is hard to say. It depends what you're searching, how you search for it, and possibly how busy the applications object model is."
Thanks for your inputs above, this is actually the point, I don't talk about environments and hardwares, It's only about the accuracy of testcomplete performance check.
Maybe tested application launch is a bad sample, let's say, I want to get performance value of a tab switch of tested application, if I do it manually it's happens immediately. If I perform test below, the performance check(100 minisec.) will never pass, because testcomplete object spy also need time to search for the tab object, this search time is involved in performance value, which exceeds 100 minisec. and make check fail.
Kind Regards,
Lee
1. click object "Tabpane2" to switch tab
2. aqPerformance.Start()
3. wait until tab2 is visible
4. aqPerformance.Check(100)
Fair point.
If you want proper, granular, mesurements, you might be better off speaking to the dev team and seeing if they can insert timestamps as property values somewhere which are triggered when they hit certain parts of the code. (Which you know relate to population of the new screen)
Then you can just extract and use them at your leisure, no need to worry about how long a search takes etc etc.
I do something similar with browser based load tests. It's about as accurate a measurement as you can get and based on the most true to life possible replication.
Of course, this is all dependent on having a helpful dev team on hand.
leeasdf9 wrote:
I want to get performance value of a tab switch of tested application, if I do it manually it's happens immediately. If I perform test below, the performance check(100 minisec.) will never pass, because testcomplete object spy also need time to search for the tab object, this search time is involved in performance value, which exceeds 100 minisec. and make check fail.
Which is why I go back to what I mention: that the needs to test automation to be able to accurately detect, interact with, and verify such interaction for any action against the application under test is in conflict with the level of performance testing you are looking to get out of TestComplete. The simple fact that TC needs to find the object will take a measurable fraction of time which is an artifact, not of the application under test, but of the requirements of test automation. So, using TestComplete to make these kinds of performance measurements is like using a screwdriver as a crowbar... it may get the job done, but there are better tools for the job with less risk of personal injury.
Now, if you go with what AlexKaras mentions and treat it as statistical analysis, that might be the way to go. You know, in fact, that the performance is VERY fast when not running under TC automation. So, you've already done a very basic performance evaluation. So, any timing that you then get from TC, with analyzing means and deviations, etc., you would then use as your baseline to take to your developers and say, "Can we make this faster?"
And that's when you get to your next problem: What EXACTLY outside of the TC environment needs to be improved in order to make it faster. Such visibility is not directly available within the TC automation so you'll have to work with some other tool. Again, AlexKaras has pointed out AQTime Pro as a good investigation tool to check for where those performance hits are happening. If you want to communicate fully to your development staff a) what is taking performance hits and b) what processes are causing the problems, that's the better tool for the job.
Hi Martin,
" The simple fact that TC needs to find the object will take a measurable fraction of time which is an artifact, not of the application under test, but of the requirements of test automation. "
Many thanks for your valueable inputs and advices, this is exactly the answer i want :)
