Friday, February 17, 2006

Automation of Setup programs ..

Let me make a blanket statement - My first impression and view is that "Setup programs" in general are not suitable for automation. They are best tested manually - unless you work for company like InstallShield whose products themselves help in creating setup programs. Those seup programs having 5-6 steps and taking just folder name as input - are not going provide return on investment for automation.

You decide to play devil's advocate and say -"I don’t agree with you" and insist on automation - Here is one approach.

1. Identify how big is setup program - how many steps are there? 10? 20? or more than 20? what are the variations possible? 100+? If yes - automation will help you.

2. Now identify the most dense and cluttered screen/step in setup that takes large number of inputs. Automate that screen only and proceed and automate let us say top 5 critical screens with 5 scripts.

3. In my opinion - there is no need to automate the setup flow unless there are more than 20 steps and 100+ variations possible.

Look at the beauty and usefulness of such analysis - you are trying to do automation, in the process, you ask so many questions. Think and create those scenarios which otherwise would never have been explored if you are following a structured and scripted test plan. See the value here. At the end you may or may not automate all those scenarios but while trying to automate and tying to convince that automation is way to go - you have tested and enriched test scenarios.

To quote Michael Bolton (, a friend and mentor - "Often automation, in itself may not lead to good testing or value directly but during the course of preparation whatever the analysis you do and questions you ask "How can I verify this"?”Why should I automate this?" "What can go wrong here" - are VALUABLE and should be done.

How? By always playing a devils advocate - "why" and "why not" If you stop questioning and just accept what is being told to you - you stop learning and cease to become a Tester...


No comments: