Forum Discussion

josh_owen's avatar
josh_owen
Occasional Contributor
10 years ago

AQtime within Test Complete

Does anyone use Test Complete with AQtime? We have experimented a little with code coverage and performance profiling, but have not found any good way to do performance testing. Recently we have been asked to incorporate AQtime into our tests that run windows services and gather performance metrics like memory and cpu.

Please comment if you have any experience using TC with AQtime performance or allocation profilers. We are just looking for some ideas for any direction.

Thank you.

  • AlexKaras's avatar
    AlexKaras
    10 years ago

    Hi,

     

    Unfortunately, I did not participate in a project where such testing was required, so just my general thoughts (that you might well be aware of)...

     

    First, I would go through the http://support.smartbear.com/viewarticle/72505/ and http://support.smartbear.com/viewarticle/67843/ for the general TestComplete-AQtime integration setup.

     

    Also, some recordings from http://support.smartbear.com/screencasts/aqtime/ page might appear to be useful.

     

    Second, while I think that you may obtain some useful results with the Coverage (and Allocation) profilers, I am doubting about the Performance one. The reason is that I think that performance profiling is a highly modular and iterative manual process.This means that you should start with defining the areas of your application where performance is most critical and start from here. For example, if your application is mostly used to enter some payments and also creates a quarter report on them, there is no big reason to pay a big attention to the report generation performance. Instead, the focus on performance of the payment registration seems to be much better idea. So, you are measuring the performance of the payment registration and look for the most poorly performing code functions. Note, that it is quite possible that at this stage you will not be interested in exact time spent in this or that function, but instead just look for the top 5 slowest functions. After identifying the slowest functions, you may talk to developers and they will correct something. After obtaining a new build, you will repeat exactly the same test and compare obtained result with the previous one to check if there was some performance improvement. If it was, then you will identify another top 5 slowest functions and repeat the cycle until the overall performance of the payment saving functionality fits the performance requirements. This is the place where you may be interested in exact timings, but remember, that the exact figures are likely to deviate between different runs, so you should repeat the measurement several times and consider some average value.

     

    Note, that for the (performance) profiling using TestComplete and AQtime, you might need a powerful computer, depending on the size of the tested application. If the tested application is big, both TestComplete and AQtime will have to keep a lot of internal information about the tested application and also have enough resources (CPU, memory and disk) to keep the measurement results without significant performance degradation.

  • AlexKaras's avatar
    AlexKaras
    Champion Level 3
    10 years ago

    Hi,

     

    Unfortunately, I did not participate in a project where such testing was required, so just my general thoughts (that you might well be aware of)...

     

    First, I would go through the http://support.smartbear.com/viewarticle/72505/ and http://support.smartbear.com/viewarticle/67843/ for the general TestComplete-AQtime integration setup.

     

    Also, some recordings from http://support.smartbear.com/screencasts/aqtime/ page might appear to be useful.

     

    Second, while I think that you may obtain some useful results with the Coverage (and Allocation) profilers, I am doubting about the Performance one. The reason is that I think that performance profiling is a highly modular and iterative manual process.This means that you should start with defining the areas of your application where performance is most critical and start from here. For example, if your application is mostly used to enter some payments and also creates a quarter report on them, there is no big reason to pay a big attention to the report generation performance. Instead, the focus on performance of the payment registration seems to be much better idea. So, you are measuring the performance of the payment registration and look for the most poorly performing code functions. Note, that it is quite possible that at this stage you will not be interested in exact time spent in this or that function, but instead just look for the top 5 slowest functions. After identifying the slowest functions, you may talk to developers and they will correct something. After obtaining a new build, you will repeat exactly the same test and compare obtained result with the previous one to check if there was some performance improvement. If it was, then you will identify another top 5 slowest functions and repeat the cycle until the overall performance of the payment saving functionality fits the performance requirements. This is the place where you may be interested in exact timings, but remember, that the exact figures are likely to deviate between different runs, so you should repeat the measurement several times and consider some average value.

     

    Note, that for the (performance) profiling using TestComplete and AQtime, you might need a powerful computer, depending on the size of the tested application. If the tested application is big, both TestComplete and AQtime will have to keep a lot of internal information about the tested application and also have enough resources (CPU, memory and disk) to keep the measurement results without significant performance degradation.