A familiar question from Management

As you have already mentioned, this question can't be answered simply, and the answer depends on many factors.

The best way in your case would be to automate several manual scenarios and measure how much time it takes you to create an automated test comparing to run it manually. I strongly recommend to multiply the time you get by 1.5-2 :)



For example, in my case if I have a 15-minutes manual test, then it takes me ~1 hour to automate it, providing that all necessary common code exists and I only need to create the test itself. So, here multiplier is equal to 4. However, if during creating the test I need to additionally create some common code, then multiplier may increase up to 10 (because common code must be reusable and therefore common code implementation is more difficult).



Lets assume that we created the test and added it to the test suite. During several first runs we will have to spend about an hour to stabilize the test. And finally 1 additional hour per month should be taken into account for maintaining such test.



That's how I calculate the time for myself, you own experiments' results might be quite different).

Hi Mike,



Another (or complimentary?) approach to those mentioned by Gennagiy might be the following one: you must make a test strategy first and answer the question: "What for and for how long do I need this test?"

If the test is intended to be executed just several times (3, 5, up to 7) and you are not going to use it for a long time (for example, you need to automate some test to concurrently access database and check if this triggers deadlock), then you may consider (keyword) record and playback approach. Most probably, you will get operational test very fast (maybe as fast as you perform the manual test) and the chances are high that it will work quite stably on the same test environment. Definitely, you are likely have false failures if execute it with different builds, on different environments, etc., but it was our original trade-off.

If your test is intended to be executed frequently, with different builds and in different environment, then you should consider standard development approach mentioned by Gennadiy - use of scripting, but not keyword record/playback, development of the common reusable code library, etc. You should make it clear for your management that in this case test automation must be considered as a development sub-project within your main one (with the clear understanding and agreement that the fact that this is a sub-project does not mean it can be treated as a low-priority one. The priorities of both these projects are the same because otherwise the development go ahead and you will not be able to do a relevant automation to cover all required areas.). Definitely, such approach means that you will have to develop and maintain the common reusable library, organize reporting, test data access, etc. but in the long term you are getting stable reliable and reusable tests that can be managed and adjusted quite easily.

Obviously, these approaches can be mixed - you can create recorded tests to quickly validate a new functionality or the functionality that is inconvenient to test manually while writing tests to be used for a long time for build acceptance and regression testing.

In my view this question is the same as any software development estimation question. I have found that this is a discipline unto itself. How long did similar projects take based on complexity and coverage? How much reusable components have we invested in? Can we reduce the time to completion of automation efforts over time by said investment? How do we calculate and demonstrate the ROI in time savings and any other metrics.