Using HISUtils.StopWatch in keyword tests

There will always be a slight delay when searching for or waiting for an object as TestComplete needs to refresh the object tree in the background to ensure it gets the object properly. Unfortunately, as far as I know, there's no easy way around it.

Hi,

I understand that is delay is unevitable. However, question comes to my mind if it would somehow be possible to measure the time TestComplete needs to refresh the object tree. If we could just take that time as well, we could extract that timeframe from the total timeframe, and then the remaining timeframe should match the time it takes for the tested application to generate the result my script is waiting for, right?



So, do you know if there is a method/property or anything, that would help me starting/stopping the stopwatch procedure, at the start/stop of the reading/refreshing of the Object Tree, to record the time TC needs to refesh the object tree?



Please, any help would be more then welcome.

Hi Mathijs,



> Is there another, more precise approach, to time more accurately?

I can think about several approaches, but it is up to you to decide what best fits your needs and budget...



a) You can use LoadComplete (http://smartbear.com/products/qa-tools/load-testing/) or load testing functionality of TestComplete (available in Enterprise edition and required additional license). It will accurately measure the time spent to get the data from the server (as we are talking about web application), but will not measure additional time spent on the client to execute client scripts, render UI controls, etc.).



b) Consider AQtime (http://smartbear.com/products/development-tools/performance-profiling/). With it you should be able to measure how much time it takes to execute the client-based scripts/code, but it will not measure transmission time between client and server. You can get the (average) total time that it takes for the end-user to see the requested page if you sum the (average) figures provided by LoadComplete and AQtime.



c) The a) and b) options provide most accurate timing. If you do need such
precision, you may consider the approach that was implemented in one of
the projects I know about: the idea was based on the educated guess (BTW, Support, correct please if the below is incorrect) that TestComplete spends extra time only when the sought for object is absent in its internal cache. After the object was found and processed by Object Spy it usually takes much less time for TestComplete to find it for the second time. Especially, if tested application state did not change significantly. With this in mind, the test code measured the figures of interest not once but several times (e.g. three times). I.e. the code opened some page measuring how much time it takes for the page to open completely (e.g. by waiting for the Visible property to get the 'true' value), then closed the page, then immediately opened and closed it two more times. The result of the first measurement then was either ignored or decreased appropriately and the average time was calculated and logged (possibly with other statistical results like deviation, etc.). Then the test code navigated to another page of interest and the measurement cycle repeated.



Hope this will inspire you with the ideas applicable for your project...

Hi Alexei,



first of all: thanks for your tips.



Actually, I am, at least at this stage, more interested on the time it takes at the client site (option B), so I will need to start exploring AQTime. Is there a demo version available so I can start to learn this?

The reason for this focus is that a colleague of mine already has got some figures on the server side (he probably used some other tool for that: not LoadComplete), so we have some server side figures at this moment. I want to retest the server side as well later on though, by using LoadComplete. However, and correct me if I am wrong: those numbers include both the server processing time as well as the data transfer time (time it takes to transfer the data (actually its pdf data) over the network. Will LoadComplete (TestComplete Enterprise) be able to distinguish the server processing time from the network transfer time? (or do I just get one 'total' number)?



As for option C, I think you could be right (sounds logical), however I used the Object Spy to select the object for the WaitProperty method. So, I guess, by using the Object Spy to "select" it, it should be in the TestComplete namemapping model, right?? Or is there anything I am missing here/need to check?



Please advice.



Thanks,

Mathijs

b) Consider AQtime (http://smartbear.com/products/development-tools/performance-profiling/). With it you should be able to measure how much time it takes to execute the client-based scripts/code, but it will not measure transmission time between client and server. You can get the (average) total time that it takes for the end-user to see the requested page if you sum the (average) figures provided by LoadComplete and AQtime.

I have downloaded and installed a demo version of AQtime. However, since this is completely new for me, I have a basic question. Since the application I am testing on, is a webapplication (it runs in the browser combined with a Java JRE instance), I was guessing that I have to select "iexplore.exe" as the module to work with (after I have selected to start a new project). Is this correct? Also, when I did that it tells me that there is no debug information, and I do not have any "cycles" to work with. I also added the Java JRE on my machine (c:\program files\java\bin\javaw.exe), but that also gives me the warning that there is no debug info available.....

Seems like I am already stuck... What am I doing wrong, and what's the best way to move forward?

Hi
Mathijs,



> [...] so I will need to start exploring AQTime. Is there a demo version available so I can start to learn this?

Yes, as with all products by AutomatedQA - http://smartbear.com/products/development-tools/performance-profiling/sign-up/ (accessible from http://smartbear.com/products/development-tools/performance-profiling/).



> [...] the application I am testing on, is a webapplication [...]

I would recommend the http://smartbear.com/support/viewarticle/17953/ and related help articles.

Also: http://smartbear.com/support/screencasts/aqtime/profiling-scripts/ (from the http://smartbear.com/support/screencasts/aqtime/ page)

and http://smartbear.com/support/viewarticle/8748/



> [...] those numbers
include both the server processing time as well as the data transfer
time (time it takes to transfer the data (actually its pdf data) over
the network.

Yes. Actually LoadComplete measures several times (like Time To First Byte, Time To Last Byte, etc.) but obviously (as LoadComplete is running on the client but not on the server box) they all consist of the time for the request to get to the server, time spent by the server to process the request and time for the response to get back to the client. However, LoadComplete can connect to the Performance Counters on the server and I am sure (did not do this personally, but pretty much sure this is possible) that by selecting proper counter it should be possible to measure time it takes the server to process the request. Subtracting this time from the overall time for the request-response roundtrip you should be able to get separate figures for the server processing and network transfer times.



>
So, I guess, by using the Object Spy to "select" it, it should be in the
TestComplete namemapping model, right??

On the one hand I am far not sure that Object Spy and runtime share the same cache. On the other hand (according to the experience) - Object Spy has some influence on the objects 'seen' by the runtime (especially when debugging and for NameMapping/Aliases functionality).

However, it was decided to implement the approach I described because of several reasons: a) it is not that difficult to enclose some action in a loop and repeat it several times; b) The action repeated immediately changes application's state the least (note, that during this test you are not testing the functionality, but only performance. That means that it is OK for the test code not to check the functionality (e.g. whether correct record was displayed, etc.) and it is better to execute one simple action at a time, but not the sequences of actions as per business process.); c) Executing actions in a loop does not require you to manually navigate in Object Spy to all tested objects (with the risk that some of them may be removed from TC cache while navigating because of cache size, because the application object was dismissed, etc.).

There will always be a slight delay when searching for or waiting for an object as TestComplete needs to refresh the object tree in the background to ensure it gets the object properly.  Unfortunately, as far as I know, there's no easy way around it.

Just to let you know: I have experienced some with it, and actually came to a good solutiom: When I use the "While [object.property.visible] <> True " code snippet, the object(property) gets recognized very fast. Right after the visible property comes to True, I stopped the Stopwatch routine.



This gave me reliable results, and the (approx.) 2 seconds delay is not occuring anymore (seems like TC does not need to update the full name mapping scheme when you use a While routine)...



Anyway thanks for all the tips!

I would also like to see this become available as a keyword test item.

Hi Amber,





Could you please describe the requested keyword test operation's functionality in detail, so I can register your request in our DB?