Community Day 2021 - Regression Testing of Performance

[sonya_m]

Let's move on to the next session of the day. Not to miss any sessions, subscribe to the event tag - #CommunityDay2021. We will be posting great content for the entire day today!

Regression Testing of Performance

by Alexandr Gubarev, TestComplete Senior Test Lead

The session takes a really deep dive into the subject. This talk is based on the experience of the TestComplete and ReadyAPI QA teams when they had decided to rapidly extend test suites to protect users from performance regressions.

This video will be useful for those who want to add performance checks to their functional tests, regardless of the type of application: Desktop, Web, Mobile, or even a web service. We will define a common approach when and for which scenarios you need to do regression performance testing, tell you about the useful features of TestComplete, learn advanced Google Sheets practices, and share recipes for quickly implementing performance checks in your tests.

Become a Community Day Winner!🏆

Watch the video sessions, post your questions and give Kudos to get the event points!

Read more about participation rules.

Watch the session:

The VideoRecorder extension developed by Alexandr mentioned in the interview.

Timestamps:

00:00 Speaker introduction

03:22 About regression and performance testing

04:45 Example of the case

06:04 Defining scenarios

09:28 Steps of tests

10:58 Using the Google Sheets for POC

11:53 TestComplete: aqPerfomance object

13:22 TestComplete: Time with Children in Log

15:15 TestComplete: Performance Counters monitoring

16:26 Demo: defining of the scenario

17:15 Demo: high-level steps

18:12 Demo: measuring

20:13 Demo: Google Sheets as storage

21:04 Demo: report table

22:43 Demo: creating web service on Google Sheets

24:51 Demo: how to publish your report for manager

25:22 Demo: work with web service from TestComplete

26:38 Demo: tests run

29:37 Some final words

Any questions on regression testing of performance? Ask away in the comments below!

[AGubarev]

@Lee_M 

  aqTestCase.Begin("Name 1");
  Log.Message(aqTestCase.CurrentTestCase.Name)
  aqTestCase.End();

[Lee_M]

Thanks,

This has a limitation, this would only work if your case was in a test case using the EP and will not work for simple tests (or testing when building the tests)

If you call the name of a case then I have found it needs to be hardcoded, you can't ready the initiator of the current test

e.g.

- order_book_test

(test sections)

1, login

   1.1, navigate - (say which function test is going to this page, ref will get login and not order_book_test)

2, order book "testing"

3, logout

[AGubarev]

>this would only work if your case was in a test case using the EP

what does "EP" mean?

>you can't ready the initiator of the current test

Sorry, didn't catch the idea.

So, you have an example:

The test case name : order_book_test
Steps:
1. Login
1.1. Navigate
1.2. Pass auth
2. Make order
3. Logout

Do you want something like this?

function main()
{
  let scenarios = ["order_book_test", "order_glass_test"]
  for (var i=0; i<scenarios.length; i++)
  {
    aqTestCase.Begin(scenarios[i]);
    Login();
    Make_Order(scenarios[i].split("_")[1]);
    Logout();
    aqTestCase.End();
  }
}

function Login()
{
  Log.AppendFolder(arguments.callee.name);
    Log.Message("Navigate");
    Log.Message("Pass auth");
  Log.PopLogFolder();
}

function Make_Order(goods)
{
  Log.AppendFolder(arguments.callee.name + ": " + goods);
    Log.Message("Make order")
  Log.PopLogFolder();
}

function Logout()
{
  Log.AppendFolder(arguments.callee.name);
    Log.Message("Logout")
  Log.PopLogFolder();
}

[Lee_M]

The case above is calling the test name from an array, whichever function outputs this name, it will always be correct.

I mean call the test name from a function that is not a direct descent of the parent

EP - Execution Plan

[AGubarev]

function main()
{
  let scenarios = ["order_book_test", "order_glass_test"]
  for (var i=0; i<scenarios.length; i++)
  {
    aqTestCase.Begin(scenarios[i]);
    Login();
    Make_Order();
    Logout();
    aqTestCase.End();
  }
}

function Login()
{
  function navigate()
  {
    Log.Error("Unable to navigate")
  }
    
  navigate();
  Log.Message("Pass auth");
}

function Make_Order(goods)
{
    Log.Message("Make order")
}

function Logout()
{
    Log.Message("Logout")
}

function EventControl1_OnLogError(Sender, LogParams)
{
  Log.Warning("An error has been fired in scenario "+aqTestCase.CurrentTestCase.Name)
}