cancel
Showing results for 
Search instead for 
Did you mean: 

Maximizing TestComplete to build efficient scripts

SOLVED
Highlighted
Occasional Contributor

Maximizing TestComplete to build efficient scripts

Hi everyone,

 

We're starting to increase the number of people handling TestComplete scripts in our company and I just wanted to read/know about how people develop their cases in TestComplete. I came across this interesting post (https://community.smartbear.com/t5/TestComplete-General-Discussions/Non-standard-ways-of-using-TestC...) from long ago and would like to have more insights/suggestions from what people do since I'm pretty sure the userbase of the tool has increased over the years.

 

As a background, we are currently using a data driven approach to testing, one test case scenario is one row in a data loop. Basically an actual test case accepts one set of test data and does actions based on those. So far it's been working great for us. We build small keyword tests that does one specific job similar to a function described in the post I have indicated above.

 

It makes it easier to make new scenarios/test cases for new features that comes in; drag and drop keyword tests that have been developed before, make new ones to do actions/checkpoints for new feature and voila a new regression case for the new feature is born. Before I hand this off and share this practice to more people in the team I have a few questions which I've had difficulty answering by myself and wanted to get people's opinions. So here they are:

 

1. Is it more efficient to make smaller keyword tests (as granular as possible, i.e one to fill up the subject field of an email reply) rather than making midsize keyword tests (i.e. one keyword test that fills up subject, body, signature, recipeints of an email reply)? This question leads to a long term dilemma I've been having with our project, is it better to have 10 KWTs each one doing small parts with no overlap of scope or is it better to have 1 KWTs doing a big chunk of work?

 

2. Is scripting more efficient than using KWTs? I've read some posts/approaches where they use KWTs to handle data, leveraging the data driven loops and data tables one can inject inside a KWT but the building blocks of test cases are all scripts, basically a KWT calling scripts per step. Right now we're leaning heavily on making KWTs the building blocks but having someone with scripting experience make complement scripts that "empowers" the KWTs.

 

3. Is there a proper way of making test data tables? We've been slowly moving away from 100 column tables where each of those columns are used to have both checkpoint data and configuration data needed to determine a scenario. The way we've been doing it the past few months was to split the big data table to smaller ones having specifc purpose for each (i.e. one for user info/permissions, one for languages available, etc.) and using "keys" from the main table to look for the necessary content in the smaller tables. 

 

If there are other posts that have discussed this, I would be interested in knowing about them. Thanks!

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted
Community Hero

Re: Maximizing TestComplete to build efficient scripts

Here's my take on that:

 

Selenium, while good, you have to do EVERYTHING in code... I mean, EVERYTHING.  Object identification, building classes, wrapping things for a unit test to include in some testing tool, etc.  EVERYTHING.

 

TestComplete:

ObjectIdentifciation -> NameMapping

Classes -> Code units/KWT

Unit tests -> Test items, projects, etc

Reports/Logs -> TestComplete logging

 

A lot of the stuff you have to do manually in Selenium, TestComplete has built in.  

 

Other advantage: Selenium is web only.  So, if you need to mix your tests between Desktop, API, Mobile, and Web, you need multiple tools.  TestComplete allows you to do ALL of them in one tool...or incorporate a few other tools into that one tool.  That Web Store that I mentioned above?  That was half of the pipeline of a ticket transaction.  There were reports, fulfillment, access control scanning, etc., done by other application platforms including desktop, kiosk, and mobile.  So, TestComplete allowed an all-in-one tool.

 

Finally... Selenium is open source... which means that if you need support... well, you can hope you find the right guy in the right forum who knows what they're doing.  TestComplete has this space and their own dedicated customer service staff.  

 

Again.. my opinion... others may vary but hopefully this helps in your decision processing.


Robert Martin
[Community Expert Group]
Please consider giving a Kudo if I write good stuff
----

Why automate?  I do automated testing because there's only so much a human being can do and remain healthy.  Sleep is a requirement.  So, while people sleep, automation that I create does what I've described above in order to make sure that nothing gets past the final defense of the testing group.
I love good food, good books, good friends, and good fun.

Mysterious Gremlin Master
Vegas Thrill Rider
Extensions available

View solution in original post

5 REPLIES 5
Highlighted
Community Hero

Re: Maximizing TestComplete to build efficient scripts

I'll make an attempt to answer based upon my experience building a similar framework based upon something I did in the past along with some recent work in building something else with a slightly different approach.  These are my opinions and your mileage may vary.

 

So, a TL;DR - I think the way you're going is pretty good.  I'd say go granular, use Scripts more than KWT, and divide up your data "smartly".  Now, on to the full details...

 

1) In both my effforts, we went more granular.  Basically, each "step" of a test was a different piece of script code with parameters to dictate the salient information for executing the step.  We have a framework I'm using now where each row in a data table indicates to execute a full test and that works, but it means that you are maintaining whole tests, not just small pieces.  I prefer going granular because in an entire test, it's rarely the full outcome that needs to be maintained but usually some small piece like navigating to a scren, selecting a particular button, etc.  Now, I didn't go so far down as having my own generic "ClickButton" methods that took on-screen objects and performed clicks.  For example, one project I worked with was an online web store for selling admission tickets to an amusement park.  So, one "step" that I coded was "select tickets".  The assumption on that step was that you were already navigated to the part of the store for selecting the tickets and the parameters were specific ticket type and quantity.  The step found the type on the web page, entered a quantity, and clicked the "add to cart" button.  I had similar steps for selecting shipping methods, applying payments, etc.  Architecturally, I had two CSV files for each "test run" that I executed.  One was simply a listing of the test cases to run (test 1, test 2, test 3, etc).  The next was a mroe complete data table that contained  the steps for the test cases.  Each row had the test case ID, a step descriptor/identifier, and then another column which was a pipe delimited set of data that was parsed into parametres and values to be used for the step.  A piece of code created an instance of a class defined for each step and, when the data was read in, created an array of those instances that was then executed as the test case.  So, whenever we needd to maintain something, we never needed to maintain a whole test case, just the code for each "step".

 

2) This is kind of a matter of opinion... but I prefer scripting.  KWT are good for someone who isn't familiar with writing code as text on screen with all the syntax and organization, etc.  They are pretty powerful as is and can do a lot.  But some of the fancy code-gymnastics that I like to do to make code elegant, maintainable, and concise you need script code to do.  The framework I described briefly above was written ENTIRELY in script.  However, you could easily make each "step" a keyword test that could then be called by name rather than the class instance stuff that I did.  If your development staff for test automation is a bunch of decent coders, I'd say go script all the way.  If not, you might find a happy medium somewhere where some parts are script and someparts are KWT.

 

3) As mentioned, you can easily create a test step table like I did with a minimum number of columns.  I managed it with 5 columns: Test Step ID, Test Case ID, Test Step Descriptor, Parameter Data, Checkpoint validation.  And then another table with 3 columns: Test Case ID, Enable/Disable, Test Case Descriptor.  Two data tables and a bunch of code.  Another framework I worked on had a few more data tables used for some additional supporting data.  For example, if I needed to input contact information into a CRM, the test step table may include, in the parameters column, a cross reference to a data table of contact information to use for input.  That way you wouldn't need to have ALL the dat in the same table, you'd just cross reference out.

 

 

So...that's my input... 


Robert Martin
[Community Expert Group]
Please consider giving a Kudo if I write good stuff
----

Why automate?  I do automated testing because there's only so much a human being can do and remain healthy.  Sleep is a requirement.  So, while people sleep, automation that I create does what I've described above in order to make sure that nothing gets past the final defense of the testing group.
I love good food, good books, good friends, and good fun.

Mysterious Gremlin Master
Vegas Thrill Rider
Extensions available
Highlighted
Occasional Contributor

Re: Maximizing TestComplete to build efficient scripts

Thanks for the reply! 

 

Just out of curiousity if you are investing heavily on scripting, wouldn't it be better to actually use Selenium or a pure software testing tool? We are also at a crossroad where we are looking into other tools. I personally prefer TestComplete because it makes handling of test data much easier and the namemapping helps with the element locators. Another advantage would also be that it's friendly to some members of our team who are not comfortable with scripting. But I'm curious of what reasons other people have. Smiley Happy

Highlighted
Community Hero

Re: Maximizing TestComplete to build efficient scripts

Here's my take on that:

 

Selenium, while good, you have to do EVERYTHING in code... I mean, EVERYTHING.  Object identification, building classes, wrapping things for a unit test to include in some testing tool, etc.  EVERYTHING.

 

TestComplete:

ObjectIdentifciation -> NameMapping

Classes -> Code units/KWT

Unit tests -> Test items, projects, etc

Reports/Logs -> TestComplete logging

 

A lot of the stuff you have to do manually in Selenium, TestComplete has built in.  

 

Other advantage: Selenium is web only.  So, if you need to mix your tests between Desktop, API, Mobile, and Web, you need multiple tools.  TestComplete allows you to do ALL of them in one tool...or incorporate a few other tools into that one tool.  That Web Store that I mentioned above?  That was half of the pipeline of a ticket transaction.  There were reports, fulfillment, access control scanning, etc., done by other application platforms including desktop, kiosk, and mobile.  So, TestComplete allowed an all-in-one tool.

 

Finally... Selenium is open source... which means that if you need support... well, you can hope you find the right guy in the right forum who knows what they're doing.  TestComplete has this space and their own dedicated customer service staff.  

 

Again.. my opinion... others may vary but hopefully this helps in your decision processing.


Robert Martin
[Community Expert Group]
Please consider giving a Kudo if I write good stuff
----

Why automate?  I do automated testing because there's only so much a human being can do and remain healthy.  Sleep is a requirement.  So, while people sleep, automation that I create does what I've described above in order to make sure that nothing gets past the final defense of the testing group.
I love good food, good books, good friends, and good fun.

Mysterious Gremlin Master
Vegas Thrill Rider
Extensions available

View solution in original post

Highlighted
Community Leader

Re: Maximizing TestComplete to build efficient scripts

I agree with on most of points.

 

I use scripting in TC since TC 6.5 ......

Tried KWT but not convinced at all.

 

Now i have a very complete framework allowing me to manage quite everything (even AS/400 mainframe). We were up to 10 persons using this framework approach (using Closure way of things) on my team in my previous company.

 

But script need far more personal involvments than KWT.  In fine you are a pure developer BUT with real end-user mind. The best combination.

 

About Selenium, most of its disavantadges were removed with Katalon Studio but TC scripting is still far better. For example the object search in TC is not limited to Xpath search.

 

Could write a book on this subject but just not having the time for ..

 

 

 

Un sourire et ça repart

Highlighted
Contributor

Re: Maximizing TestComplete to build efficient scripts

Being a selenium guy now using TC:

 

TC Pros:

-Fantastic object recognition and object recorder, I manages to find anything.

-Object recognition is fast on complex xpaths, it parses way faster than any selenium driver.

-Test execution layout and control is excellent.

-Html/MHTreports are excellent. I've often been asked to provide the same reports in other environment.

-Way faster to actually write tests when you can stop/start whenever and just record the object.

 

TC Cons:

-Editing scripts in TCEditor is painful. and intellisense is almost non existant (javascript and python)

-single window editor. I use VisualStudioCode with python and javascript and just execute and run in TC editor.

-Licencing preventing you to from running lots of small scattered tests, tend to become large testsuites.

-Windows centric. We run all browser-tests in linux containers with headless FF/Chrome/MSEdge/JSDom.

-No native nodejs/npm support, would have been nice to just import modules from npm, but I've learnt lots of javascript where I often before just found a crappy npm module to do my work.