Non-standard ways of using TestComplete for building test cases

[leandropoblet]

Hi all,

I'm working on this concept to make use of TestComplete in a different way. Instead of adding test cases, steps to those test cases inside my project I'd like to have one "step" per test case so for example if I was testing a login system and I had three test cases (just an example but you'll see):

1. Open app, enter no credentials, click login.

    Expected result: no login, error message.

2. Open app, enter wrong credentials, click login.

    Expected result: no login, different error message.

3. Open app, enter credentials, click login.

    Expected result: user login. User lands on user dashboard.

Right? Well... using TestComplete as it's supposed I'd create one script for launching the app, another for dealing with TextBoxes (entering text as parameter), another for clicking in buttons (Login, Cancel), another for checking error messages when necessary and so on. Right?

What if I had only one routine call per test case as this in my project?

Test Login:

   1. Test login case 1

function TestCase1()
{
 OpenApp(MyApp);
 EnterCredentials("","");
 ClickLoginButton();
 CheckErrorMessage("Invalid Username or Password");
}

   2. Test login case 2

function TestCase2()
{
  OpenApp(MyApp);
  EnterCredentials("1234","1234");
  ClickLoginButton();
  CheckErrorMessage("Invalid Password for user 1234");
}

   3. Test login case 3

function TestCase3() 
{
 OpenApp(MyApp);
 EnterCredentials("1234","4321");
 ClickLoginButton();
 CheckUserHasLoggedIn("Welcome back! Last login 16.12.2016 at 16:10pm");
}

So in that case I'd be calling one function and each function would call the right functions every time. 

As I see it the benefit would be not having to maintain the test cases steps in my project but only those functions. 

I have to say that I'm a lot faster using the keyboard than clicking and selecting, etc. So this would allow me to build test cases much faster.

The question is: does this kind of usage have any penalties as memory consumption, logging issues or anything that is really undesirable?

Isn't it a faster, more robust way of coding my test cases? Considering the time it takes to build a test case the traditional way, the lack of support for copying test cases between projects (even within the same project).... wouldn't this solve those issues?

I'd like to hear your ideas and any other "non-standard" way of Using TestComplete. Specially those that make you life easier and your testings more accurate/faster.

Much appreciated,

Leo

[baxatob]

Hi,

You provide a good approach, however I am missing why you have called it "non-standard". On the contrary it should be a standard way.

I propose to go further and upgrade the code, to make login/authorization procedures as a separate function:

function StartApplication(login, password)
{
    OpenApp(MyApp);
    EnterCredentials(login, password);
    ClickLoginButton();
}

Now the test case looks more simple and robust:

function TestCase1()
{
    StartApplication(login, password);
    CheckErrorMessage("Invalid Username or Password");
    StopApplication();
}

and so on...

The general idea is to reduce the amount of duplicated code.

[tristaanogre · Accepted solution]

I'm with @baxatob... what you've described is not exactly a non-standard way of using TestComplete.  Your original post describes, essentially, what I started doing years ago with POS product that I was testing.  Logging in was a function, navigating to the sales screen was a function, selecting a product for sale was a function, tendering a payment was a function, etc.  Then, creating a test case was just using the building blocks to put together different test scenarios into separate functions.  

This is actually, one of the preferred methodologies in automated testing, not just with the TestComplete tool.  You modularize your code (even if you're doing KeywordTests, you can do this) and then build your test cases out of those modules.  It is actually one step away from going with something more data-driven in a framework where your data consists of the scenarios and the test "steps" to be executed and then you run a driver to read the data and execute the steps based upon the data source.  Changing a test case in that type of setup is then simply a matter of adjusting the data to reflect the desired changes.  Adding a test case is simply adding new rows to your data and so on.  

What a lot of people do when it comes to using TestComplete to create test cases is all the code for a test case is contained within the test case... which ends up with a lot of duplicated code all over the place and a maintenance nightmare. As @baxatob said, going modular like you are reduces the duplicated code.  But even having several test cases like TestCase1, TestCase2, TestCase3, each one calling "StartApplication" is, effectively, duplicated code.  Each test case "looks" the same, essentially.  Code wise, it makes the most sense to have a function for each step and then one function to read data to execute those steps in a configured order.  Code is reduced by an exponential factor and your primary maintenance for test cases is data entry and your code maintenance is simply maintaining your step-functions as your application and testing needs change.

There are a LOT of ways of using TestComplete, some more common than others... what is probably the more pertinent question is what are "best practices" for test automation in general and how does TestComplete fit into those best practices.  You are on a good path, I think, towards utilizing TestComplete in an efficient manner and implementing those best practices.

[Marsha_R]

I realize that these are just examples but please please PLEASE don't name anything TestCase1, TestCase2, etc.  Useful descriptive names are key, especially if you are going to have many functions.  

[leandropoblet]

Hi all and thanks for your replies.

Both @baxatob and @tristaanogre got exactly the point and just made my life a lot easier! 

@tristaanogre you lost me a bit here: 

---

@tristaanogre wrote:

It is actually one step away from going with something more data-driven in a framework where your data consists of the scenarios and the test "steps" to be executed and then you run a driver to read the data and execute the steps based upon the data source.  Changing a test case in that type of setup is then simply a matter of adjusting the data to reflect the desired changes.  Adding a test case is simply adding new rows to your data and so on.  

Code wise, it makes the most sense to have a function for each step and then one function to read data to execute those steps in a configured order.  Code is reduced by an exponential factor and your primary maintenance for test cases is data entry and your code maintenance is simply maintaining your step-functions as your application and testing needs change.

---

I'm starting today turning those functions into modules and calling them. I though this was closer to a bad practice rather than a good one.

Thank you again for this massive game changer for me.

@Marsha_R Yes you got it right. It was on the go and to make it very obvious to anybody reading the post what was the intention behind. I'll try and come with a better name next time. 

[Colin_McCrae]

I take it a step further than TestComplete test cases.

I use a keyword and data driven framework. Each input line calls a function. I guess that would be a test "step". Although, it's not really a TestComplete "step" in the true sense of the word as one function may generate several entries to the TestComplete log. But I also do my own custom logging and reporting, so this doesn't really matter.

But then I enclose a series of these "steps" within calls (which are pretty much TFS Start <Case ID Date> and TFS End) to TFS. This ties a series of "steps" to a test "case" as set out on our test management system.

Makes sense to do it this way as most "steps" can be re-used many times over numerous "cases".

In my case, the majority of the "steps" are calls to handler functions. And the handler functions each deal with a certain type of control. There are other calls the "steps" can make. These tend to be slightly more specialised and/or application specific. But the majority of the handler functions can also be re-used on any other application written in the same language as they address controls directly. The location, type of control, action to take, data required, and expected results are all provided by the input data.

Gives me maximum flexibility and re-use.

[tristaanogre]

---

@leandropoblet wrote:

@tristaanogre you lost me a bit here: 

---

Sorry. 🙂 

What I did was describe to you the framework that I developed a bit ago.  Nothing huge but certainly a way of doing things that you might have considered "non-standard".  In essence, what @Colin_McCrae described is close to what I'm doing but not quite to the granular level that he has.  I presented it in a webinar recently that you can view here but I do not present it as the only way to do such things. Basically, instead of a routine like

function sellSomething() {
    Login('user', 'password');
    Navigate('sales screen');
    SelectProduct('item 1');
    TenderPayment('cash', '20.00');
    Logout();
}

I have a set of data tables.  One table is the list of test cases indexed by an integer.  The other table is a list of "steps" that are associated with those test cases.  The function above would be one of those test cases.  Each module call in it would be  a step in the second table. Using DDT objects, I iterate through the first table, grab a test case, then build in memory the list of "steps" from the second table and then run them in order. I record the results, dispose of the data in memory, and go to the next set of records. So, the code itself consists of those individual modules and then a few routines of code to read the data and execute the modules. Changing a test case, as mentioned, is then not a matter of changing lines of code but simply changing the information stored in the data tables.

[mgroen2]

@leandropoblet First of all, good question. 

@tristaanogre and @baxatob already gave directions for you to tackle this kind of situations.

Don't take it the wrong way, if you want to gain knowledge and insight on how to design a well-thought test approach (including design of high-quality test automation frameworks), I recommend to start learning something on ISTQB material (google ISTQB)... there might be ISTQB-equivalent frameworks, though (not sure).

The good thing I like about TestComplete is that it's a very versatile product allowing both designing, building, implementation of "advanced end-to-end data driven testing frameworks", as well as "coding something the quick and dirty way around procedure X to make it looks like it's being tested".