Pro-Tip: Automation (or testing in general) is NOT "Real World"

[tristaanogre]

One of the things that business analysts and software testers sometimes get hung up on is whether or not a test case, when executed, accurately exercises the software as it would be in the "real world." To some extent, there is a place for "real world" testing.  Does everything work EXACTLY in the same flow and timing that would happen in the real world? If I set up a trigger to happen in 24 hours, if I wait 24 hours, will the trigger activate?  That kind of makes sense.

But I would suggest that at LEAST 90% of the time, when testing software, the goal is not to make sure it works like "real life" but that, when given a particular set of inputs, the output matches an expected result.  And that means, sometimes, you "fake" the inputs to make them LOOK like real life so that you can get expected outputs.

Let's say, for example, you have a piece of software that, if something is entered so that, based upon the date/time stamp of the entry, something else shows up in 2 hours, do you REALLY need enter the information as if it happened NOW and then wait 2 hours?

The answer: no.  If I enter data into the software where the date/time stamp is 2 hours in the past, then the code SHOULD work the same.  If I'm doing an order entry system where a task needs to be done 2 hours after the order date/time, if I place the order so that it LOOKS like it happened 2 hours ago, then when I go into the software, I should see my 2 hour task.

Now, of course, some of this is dependent upon the software itself, but the point should be obvious.  You have a piece of software that, given a data condition will do something.  If you can simulate that data condition, then the software should STILL do that same thing.

This plays out ENORMOUSLY in test automation.  You may not always be able to 100% simulate what happens in "real life".  In those cases, that's when the creativity of the tester comes into play.  What can you do to simulate the required data condition? Whatever it is, even if it's not "real life", it still exercises the code under test by verifying that the expected output happens given the expected input.

So, next time your test case says, "Enter data and wait 24 hours", please, whatever you do, do NOT add a Delay(86400000) (that's the number of milliseconds in 24 hours, by the way) to your test case.  There usually is a better way to generate the required data.

[Marsha_R]

Agreed!

Also, real users are never going to type as fast as TestComplete does, so timing issues you find may not be worth the research time it would take to fix them.  Slowing your test down a little is usually fine.

Code coverage is another area that can be misleading.  Okay, you have 80% or 90% code coverage, but do all those areas of the code get used or changed frequently?  If you are working on code that has been around a long time, then you may be spending time creating tests for things that affect very few users.