Forum Discussion

wmtan01's avatar
wmtan01
Contributor
5 years ago

Maximizing TestComplete to build efficient scripts

Hi everyone,

 

We're starting to increase the number of people handling TestComplete scripts in our company and I just wanted to read/know about how people develop their cases in TestComplete. I came across this interesting post (https://community.smartbear.com/t5/TestComplete-General-Discussions/Non-standard-ways-of-using-TestComplete-for-building-test-cases/m-p/133021#M24816) from long ago and would like to have more insights/suggestions from what people do since I'm pretty sure the userbase of the tool has increased over the years.

 

As a background, we are currently using a data driven approach to testing, one test case scenario is one row in a data loop. Basically an actual test case accepts one set of test data and does actions based on those. So far it's been working great for us. We build small keyword tests that does one specific job similar to a function described in the post I have indicated above.

 

It makes it easier to make new scenarios/test cases for new features that comes in; drag and drop keyword tests that have been developed before, make new ones to do actions/checkpoints for new feature and voila a new regression case for the new feature is born. Before I hand this off and share this practice to more people in the team I have a few questions which I've had difficulty answering by myself and wanted to get people's opinions. So here they are:

 

1. Is it more efficient to make smaller keyword tests (as granular as possible, i.e one to fill up the subject field of an email reply) rather than making midsize keyword tests (i.e. one keyword test that fills up subject, body, signature, recipeints of an email reply)? This question leads to a long term dilemma I've been having with our project, is it better to have 10 KWTs each one doing small parts with no overlap of scope or is it better to have 1 KWTs doing a big chunk of work?

 

2. Is scripting more efficient than using KWTs? I've read some posts/approaches where they use KWTs to handle data, leveraging the data driven loops and data tables one can inject inside a KWT but the building blocks of test cases are all scripts, basically a KWT calling scripts per step. Right now we're leaning heavily on making KWTs the building blocks but having someone with scripting experience make complement scripts that "empowers" the KWTs.

 

3. Is there a proper way of making test data tables? We've been slowly moving away from 100 column tables where each of those columns are used to have both checkpoint data and configuration data needed to determine a scenario. The way we've been doing it the past few months was to split the big data table to smaller ones having specifc purpose for each (i.e. one for user info/permissions, one for languages available, etc.) and using "keys" from the main table to look for the necessary content in the smaller tables. 

 

If there are other posts that have discussed this, I would be interested in knowing about them. Thanks!

  • Here's my take on that:

     

    Selenium, while good, you have to do EVERYTHING in code... I mean, EVERYTHING.  Object identification, building classes, wrapping things for a unit test to include in some testing tool, etc.  EVERYTHING.

     

    TestComplete:

    ObjectIdentifciation -> NameMapping

    Classes -> Code units/KWT

    Unit tests -> Test items, projects, etc

    Reports/Logs -> TestComplete logging

     

    A lot of the stuff you have to do manually in Selenium, TestComplete has built in.  

     

    Other advantage: Selenium is web only.  So, if you need to mix your tests between Desktop, API, Mobile, and Web, you need multiple tools.  TestComplete allows you to do ALL of them in one tool...or incorporate a few other tools into that one tool.  That Web Store that I mentioned above?  That was half of the pipeline of a ticket transaction.  There were reports, fulfillment, access control scanning, etc., done by other application platforms including desktop, kiosk, and mobile.  So, TestComplete allowed an all-in-one tool.

     

    Finally... Selenium is open source... which means that if you need support... well, you can hope you find the right guy in the right forum who knows what they're doing.  TestComplete has this space and their own dedicated customer service staff.  

     

    Again.. my opinion... others may vary but hopefully this helps in your decision processing.

  • tristaanogre's avatar
    tristaanogre
    Esteemed Contributor

    I'll make an attempt to answer based upon my experience building a similar framework based upon something I did in the past along with some recent work in building something else with a slightly different approach.  These are my opinions and your mileage may vary.

     

    So, a TL;DR - I think the way you're going is pretty good.  I'd say go granular, use Scripts more than KWT, and divide up your data "smartly".  Now, on to the full details...

     

    1) In both my effforts, we went more granular.  Basically, each "step" of a test was a different piece of script code with parameters to dictate the salient information for executing the step.  We have a framework I'm using now where each row in a data table indicates to execute a full test and that works, but it means that you are maintaining whole tests, not just small pieces.  I prefer going granular because in an entire test, it's rarely the full outcome that needs to be maintained but usually some small piece like navigating to a scren, selecting a particular button, etc.  Now, I didn't go so far down as having my own generic "ClickButton" methods that took on-screen objects and performed clicks.  For example, one project I worked with was an online web store for selling admission tickets to an amusement park.  So, one "step" that I coded was "select tickets".  The assumption on that step was that you were already navigated to the part of the store for selecting the tickets and the parameters were specific ticket type and quantity.  The step found the type on the web page, entered a quantity, and clicked the "add to cart" button.  I had similar steps for selecting shipping methods, applying payments, etc.  Architecturally, I had two CSV files for each "test run" that I executed.  One was simply a listing of the test cases to run (test 1, test 2, test 3, etc).  The next was a mroe complete data table that contained  the steps for the test cases.  Each row had the test case ID, a step descriptor/identifier, and then another column which was a pipe delimited set of data that was parsed into parametres and values to be used for the step.  A piece of code created an instance of a class defined for each step and, when the data was read in, created an array of those instances that was then executed as the test case.  So, whenever we needd to maintain something, we never needed to maintain a whole test case, just the code for each "step".

     

    2) This is kind of a matter of opinion... but I prefer scripting.  KWT are good for someone who isn't familiar with writing code as text on screen with all the syntax and organization, etc.  They are pretty powerful as is and can do a lot.  But some of the fancy code-gymnastics that I like to do to make code elegant, maintainable, and concise you need script code to do.  The framework I described briefly above was written ENTIRELY in script.  However, you could easily make each "step" a keyword test that could then be called by name rather than the class instance stuff that I did.  If your development staff for test automation is a bunch of decent coders, I'd say go script all the way.  If not, you might find a happy medium somewhere where some parts are script and someparts are KWT.

     

    3) As mentioned, you can easily create a test step table like I did with a minimum number of columns.  I managed it with 5 columns: Test Step ID, Test Case ID, Test Step Descriptor, Parameter Data, Checkpoint validation.  And then another table with 3 columns: Test Case ID, Enable/Disable, Test Case Descriptor.  Two data tables and a bunch of code.  Another framework I worked on had a few more data tables used for some additional supporting data.  For example, if I needed to input contact information into a CRM, the test step table may include, in the parameters column, a cross reference to a data table of contact information to use for input.  That way you wouldn't need to have ALL the dat in the same table, you'd just cross reference out.

     

     

    So...that's my input... 

    • wmtan01's avatar
      wmtan01
      Contributor

      Thanks for the reply! 

       

      Just out of curiousity if you are investing heavily on scripting, wouldn't it be better to actually use Selenium or a pure software testing tool? We are also at a crossroad where we are looking into other tools. I personally prefer TestComplete because it makes handling of test data much easier and the namemapping helps with the element locators. Another advantage would also be that it's friendly to some members of our team who are not comfortable with scripting. But I'm curious of what reasons other people have. :smileyhappy:

      • tristaanogre's avatar
        tristaanogre
        Esteemed Contributor

        Here's my take on that:

         

        Selenium, while good, you have to do EVERYTHING in code... I mean, EVERYTHING.  Object identification, building classes, wrapping things for a unit test to include in some testing tool, etc.  EVERYTHING.

         

        TestComplete:

        ObjectIdentifciation -> NameMapping

        Classes -> Code units/KWT

        Unit tests -> Test items, projects, etc

        Reports/Logs -> TestComplete logging

         

        A lot of the stuff you have to do manually in Selenium, TestComplete has built in.  

         

        Other advantage: Selenium is web only.  So, if you need to mix your tests between Desktop, API, Mobile, and Web, you need multiple tools.  TestComplete allows you to do ALL of them in one tool...or incorporate a few other tools into that one tool.  That Web Store that I mentioned above?  That was half of the pipeline of a ticket transaction.  There were reports, fulfillment, access control scanning, etc., done by other application platforms including desktop, kiosk, and mobile.  So, TestComplete allowed an all-in-one tool.

         

        Finally... Selenium is open source... which means that if you need support... well, you can hope you find the right guy in the right forum who knows what they're doing.  TestComplete has this space and their own dedicated customer service staff.  

         

        Again.. my opinion... others may vary but hopefully this helps in your decision processing.