Ask a Question

Pro-Tip: Automation (or testing in general) is NOT "Real World"

Esteemed Contributor

Pro-Tip: Automation (or testing in general) is NOT "Real World"

One of the things that business analysts and software testers sometimes get hung up on is whether or not a test case, when executed, accurately exercises the software as it would be in the "real world." To some extent, there is a place for "real world" testing.  Does everything work EXACTLY in the same flow and timing that would happen in the real world? If I set up a trigger to happen in 24 hours, if I wait 24 hours, will the trigger activate?  That kind of makes sense.


But I would suggest that at LEAST 90% of the time, when testing software, the goal is not to make sure it works like "real life" but that, when given a particular set of inputs, the output matches an expected result.  And that means, sometimes, you "fake" the inputs to make them LOOK like real life so that you can get expected outputs.


Let's say, for example, you have a piece of software that, if something is entered so that, based upon the date/time stamp of the entry, something else shows up in 2 hours, do you REALLY need enter the information as if it happened NOW and then wait 2 hours?


The answer: no.  If I enter data into the software where the date/time stamp is 2 hours in the past, then the code SHOULD work the same.  If I'm doing an order entry system where a task needs to be done 2 hours after the order date/time, if I place the order so that it LOOKS like it happened 2 hours ago, then when I go into the software, I should see my 2 hour task.


Now, of course, some of this is dependent upon the software itself, but the point should be obvious.  You have a piece of software that, given a data condition will do something.  If you can simulate that data condition, then the software should STILL do that same thing.


This plays out ENORMOUSLY in test automation.  You may not always be able to 100% simulate what happens in "real life".  In those cases, that's when the creativity of the tester comes into play.  What can you do to simulate the required data condition? Whatever it is, even if it's not "real life", it still exercises the code under test by verifying that the expected output happens given the expected input.


So, next time your test case says, "Enter data and wait 24 hours", please, whatever you do, do NOT add a Delay(86400000) (that's the number of milliseconds in 24 hours, by the way) to your test case.  There usually is a better way to generate the required data.

Robert Martin
[Hall of Fame]
Please consider giving a Kudo if I write good stuff

Why automate?  I do automated testing because there's only so much a human being can do and remain healthy.  Sleep is a requirement.  So, while people sleep, automation that I create does what I've described above in order to make sure that nothing gets past the final defense of the testing group.
I love good food, good books, good friends, and good fun.

Mysterious Gremlin Master
Vegas Thrill Rider
Extensions available
Community Hero



Also, real users are never going to type as fast as TestComplete does, so timing issues you find may not be worth the research time it would take to fix them.  Slowing your test down a little is usually fine.


Code coverage is another area that can be misleading.  Okay, you have 80% or 90% code coverage, but do all those areas of the code get used or changed frequently?  If you are working on code that has been around a long time, then you may be spending time creating tests for things that affect very few users.  

[Community Hero]
[Community Heroes] are not employed by SmartBear Software but
are just volunteers who have some experience with the tools by SmartBear Software
and a desire to help others. Posts made by [Community Heroes]
may differ from the official policies of SmartBear Software and should be treated
as the own private opinion of their authors and under no circumstances as an
official answer from SmartBear Software.
The [Community Hero] signature is used with permission by SmartBear Software.
Showing results for 
Search instead for 
Did you mean: