Ask a Question

Community Day 2021 - Regression Testing of Performance

sonya_m
SmartBear Alumni (Retired)

Community Day 2021 - Regression Testing of Performance

Let's move on to the next session of the day. Not to miss any sessions, subscribe to the event tag - #CommunityDay2021. We will be posting great content for the entire day today!

 

Regression Testing of Performance

by Alexandr Gubarev, TestComplete Senior Test Lead

 

The session takes a really deep dive into the subject. This talk is based on the experience of the TestComplete and ReadyAPI QA teams when they had decided to rapidly extend test suites to protect users from performance regressions.

 

This video will be useful for those who want to add performance checks to their functional tests, regardless of the type of application: Desktop, Web, Mobile, or even a web service. We will define a common approach when and for which scenarios you need to do regression performance testing, tell you about the useful features of TestComplete, learn advanced Google Sheets practices, and share recipes for quickly implementing performance checks in your tests.

 

Become a Community Day Winner!πŸ†

Watch the video sessions, post your questions and give Kudos to get the event points!

Read more about participation rules.

 

Watch the session:

 

The VideoRecorder extension developed by Alexandr mentioned in the interview.

 

Timestamps:

00:00 Speaker introduction

03:22 About regression and performance testing

04:45 Example of the case

06:04 Defining scenarios

09:28 Steps of tests

10:58 Using the Google Sheets for POC

11:53 TestComplete: aqPerfomance object

13:22 TestComplete: Time with Children in Log

15:15 TestComplete: Performance Counters monitoring

16:26 Demo: defining of the scenario

17:15 Demo: high-level steps

18:12 Demo: measuring

20:13 Demo: Google Sheets as storage

21:04 Demo: report table

22:43 Demo: creating web service on Google Sheets

24:51 Demo: how to publish your report for manager

25:22 Demo: work with web service from TestComplete

26:38 Demo: tests run

29:37 Some final words

 

Any questions on regression testing of performance? Ask away in the comments below!


Sonya Mihaljova
Community and Education Specialist

24 REPLIES 24
AGubarev
SmartBear Alumni (Retired)

@AlexKaras ,

yes, we have a lot of internal things which we will be glad to share.

 

Thanks @sonya_m and @TanyaYatskovska we've had a chance to "say" something to community πŸ™‚

 

For now, we have no ready video/tutorial for that. But I found that we already have something in Community: https://community.smartbear.com/t5/TestComplete-Questions/Allure-Reporting-using-TestComplete-14-1-J...

Alexander Gubarev
TestComplete QA Engineer

Alexandr,

 

I found that we already have something in Community:

Thank you for the link! I seem to miss this thread as I don't remember it. Will read it for details.

 

For now, we have no ready video/tutorial for that.

This was a kind or request for the future πŸ™‚

 

Regards,
  /Alex [Community Champion]
____
[Community Champions] are not employed by SmartBear Software but
are just volunteers who have some experience with the tools by SmartBear Software
and a desire to help others. Posts made by [Community Champions]
may differ from the official policies of SmartBear Software and should be treated
as the own private opinion of their authors and under no circumstances as an
official answer from SmartBear Software.
The [Community Champion] signature is assigned on quarterly basis and is used with permission by SmartBear Software.
https://community.smartbear.com/t5/Community-Champions/About-the-Community-Champions-Program/gpm-p/252662
================================

There is also another way to present trends with your tests in Jenkin's.  We use an extension called 'Plots', that in our case, we use to graph page load times daily/weekly.

Off-topic:

 

after that can investigate this deeper with some additional tools like profilers: https://smartbear.com/product/aqtime-pro/features/performance-profiler/

 

Irony of life: AQTime (MemProof) was the initial product by SmartBear (AutomatedQA and TotalQA (if I remember it right)) and TestComplete was created to automate its testing (as it was said here somewhere in time).

Now AQTime is used to performance-tune TestComplete... πŸ™‚

 

I would like to believe that performance, resources usage and memory (leaks) control and usage optimization are still of significant importance for  at least some companies / projects and that AQTime is not an internal product of low importance for SmartBear.

Might it be an option to have an event like this Community Day but dedicated to AQTime sometime in the future? Maybe with some detailed and technical demos of how to use AQTime with modern .Net applications?

 

 
 
Regards,
  /Alex [Community Champion]
____
[Community Champions] are not employed by SmartBear Software but
are just volunteers who have some experience with the tools by SmartBear Software
and a desire to help others. Posts made by [Community Champions]
may differ from the official policies of SmartBear Software and should be treated
as the own private opinion of their authors and under no circumstances as an
official answer from SmartBear Software.
The [Community Champion] signature is assigned on quarterly basis and is used with permission by SmartBear Software.
https://community.smartbear.com/t5/Community-Champions/About-the-Community-Champions-Program/gpm-p/252662
================================

Thanks @sonya_m and @TanyaYatskovska we've had a chance to "say" something to community

Our sincere kudos go to Tanya, Sonya, management who approved this event and all guys who participated in the event and assisted in its preparation.

 

 

 
Regards,
  /Alex [Community Champion]
____
[Community Champions] are not employed by SmartBear Software but
are just volunteers who have some experience with the tools by SmartBear Software
and a desire to help others. Posts made by [Community Champions]
may differ from the official policies of SmartBear Software and should be treated
as the own private opinion of their authors and under no circumstances as an
official answer from SmartBear Software.
The [Community Champion] signature is assigned on quarterly basis and is used with permission by SmartBear Software.
https://community.smartbear.com/t5/Community-Champions/About-the-Community-Champions-Program/gpm-p/252662
================================
Marsha_R
Champion Level 3

Thanks for the video!  Do you have a preference when to start performance testing for a baseline of a new project.  Would you start at the beginning and see how performance is affected by each new feature or wait till the end or close to the end and then try to fix it?  I can see a case for both.

AGubarev
SmartBear Alumni (Retired)

Hi, @Marsha_R 

Thanks for the question.

From my experience, when we start test projects from scratch, first of all Devs need to "understand" and implement basic functionality. And this functionality is near to POC. So, it can be changed in any moment. So, I'd concentrate all QA resources on functional tests. Maybe the only exception, if we talk about performance, is load/stress testing. I remember the case, when we as QA perform load / stress of the the raw version of client-service app, and as result developers totally change the infrastructure and solution vendors (the previous had limited scale capabilities). So, if you test project which potentially will be scaled (it has database, or server on background) and it's not typical project for your company. It's reasonable to investigate limitations on the earlier stages.

 

Regarding to performance regression testing, I still think that QA resources are "luxury", so, product teems should use it on most critical scenarios. And it's hard to predict such scenarios if the app is not used by real customers. Of cause, there are obvious issues with performance. But from my experience even ordinary functionality tests can found them. So, to sum up, I cannot see the cases, when it reasonable to develop such kind of tests at least before the core functionality is covered with functionality tests

Alexander Gubarev
TestComplete QA Engineer

Hi Marsha,

 

Do you have a preference [...]

If I may jump in, I would suggest intermediate suggestion - start when the foundation of solution has been established and it was decided that it is good enough and may and will be used from now on (i.e. that generic architecture of solution has been tried and it was decided that architecture (core engine) is approved). This seems to be a good moment to create initial baseline and then move on and monitor how each new feature affects performance.

Does this sounds reasonable?

 

P.S. Update: It appeared that I wrote practically the same that @AGubarev replied three minutes before πŸ™‚

 

Regards,
  /Alex [Community Champion]
____
[Community Champions] are not employed by SmartBear Software but
are just volunteers who have some experience with the tools by SmartBear Software
and a desire to help others. Posts made by [Community Champions]
may differ from the official policies of SmartBear Software and should be treated
as the own private opinion of their authors and under no circumstances as an
official answer from SmartBear Software.
The [Community Champion] signature is assigned on quarterly basis and is used with permission by SmartBear Software.
https://community.smartbear.com/t5/Community-Champions/About-the-Community-Champions-Program/gpm-p/252662
================================
sonya_m
SmartBear Alumni (Retired)

This has been one of the most active threads, thanks everyone for being so involved!

The best question bonus points will go to @hannecroonen


Sonya Mihaljova
Community and Education Specialist

@AGubarev 

 

arguments.callee.caller

 

I will further investigate this approach, but last time I ran it, it got one of the subfunctions names (eg Close dialogue, which is called multiple times and is not the parent script run)

cancel
Showing results forΒ 
Search instead forΒ 
Did you mean:Β