Why GUI Test Automation is Popular and Tempting? Part I - Business context

It is rare to someone on testing in IT world not be aware of "QTP, WinRunner, Rational Robot etc ". For the most in IT world, the word automation is synonymous with GUI automation. Perhaps some even might ask - "Are there any other forms of automation other GUI automation". Lots of how IT world sees automation has to do with how they see testing itself. You might have heard this "celebrated" statement - "Regression testing is more suitable to do automation (read GUI) so that we don't have to have our IT staff do that unnecessary testing manually".

To my surprise, having been associated with IT for many years now, people here still believe in perfect requirement, perfect test case and hence "doing it right first time". In contrast to this, software product world (likes of Microsoft, Google) treats test automation and hence testing in a total different way. These two probably operate from two extreme ends of automation spectrum. While discussion how automation is perceived and realized in IT and software product world can be a separate post in itself, I propose to elaborate here my views on the popularity on GUI Automation in IT.

Testing is necessary evil and should get done quickly

This is the reason #1 that drives GUI automation in IT. Business spends on getting IT application features developed and hence often fails to see a reason why they should spend on "testing" at all. Business feels that IT should get their acts together, get aligned to business (hence IT-business alignment is often a well understood but poorly executed theme) and use the money towards delivering business value. I even have heard some business folks saying "We pay for the application features not for their testing". Hence funding testing becomes IT's problem. That is why they look for solutions to reduce testing (or spend on it). As simple as it sounds, IT pays for testing (not the business)

Testers (read business users or analysts) are hard to find and they hate regression testing

Traditionally, in IT business analysts (aka BA's) are hot commodities. They do most of what we call as "testing". With outsourcing wave sweeping IT, some of BA roles are taken by either BA's from outsourcing vendors or some junior tester working under supervision of BA through highly scripted test procedures. The problem is that these BA resources are few and are in great demand. But as software releases happen, changes come in – testing (regression testing) has to happen and BA's hate regression testing (or all of testing). So, an IT manager needs to find a way out to carry out this regression testing somehow.

How to get testing done fast and cheaper?

If you want to do something fast, give it to machine or a computer program - goes a typical approach of an IT manager. So automation could be the answer. If you position automation as an aid to reduce the cycle time to market chances of you getting funding is very high. Surprisingly this claim is so powerful that when made often goes unchallenged. The magic word "reduction of cycle time" (that too without any qualification) is tempting for many that it will make them to completely overlook bottlenecks in development (programming), bug fixing, investigation and any other non-testing activities. Thus speaking in terms of business language is so important in proliferation and apparent success of GUI automation in IT.

With GUI automation, you can easily make a business case for automation.

To me, the terms like test cases, regression testing, test cycle time etc are the ones that business stakeholders appear to understand very well. As compared to unit tests and other forms of non GUI automation, you can make your case for investment in GUI automation to business stakeholders. Let us say you have 5000 test cases for a regression testing to cover that takes 5 person days to complete, automate 50% of them you can straight way knock off 50% of your testing cycle time reduced. This claim due to its simple math and logic makes a compelling case for GUI automation or simply "automation". What else can be more appealing than investing once in automation (for the sake of making the business case and further forever – down play the maintenance costs) and saving on a percentage proportional amount of automation achieved for 100's of cycles in the future?

Tool vendors help you by talking in business language

Enter tool vendors – with the fancy looking charts, scary quotes like "more than 60% of project cost goes to testing, what are you doing to reduce it" and likes Gartner's and Forrester's to support the claims with numbers, life for an IT manager in charge of testing has never been so easy. Just buy a tool, engage an outsourcing vendor to do GUI automation – all your testing worries are over in one go. Dreaming never ends here. Take a look at any brochures, websites, commercial literatures – they are full of business terms like ROI, Cost of testing, business impact, and Time to market and so on - making a perfect connect with those that matter in making decisions about spending.

Now try doing this (or these) with non GUI tests such as API tests or xUnit framework based unit tests – what will business say? What will IT managers say – "well, developers got to do that testing anyway"? So there is no apparent business case to make so sell automation. Writing unit tests will not speedup your testing cycle, will not reduce time to market, might reduce some portion of that "boring" regression testing – but. So you have no financially oriented incentive to do any automation that is non GUI. Since GUI automation paradigm as so much hardened into dogma – very few challenge it.

If you are the one working in a software product company – you would say "do less of GUI automation, they are very brittle and costly to maintain". You would be surprise to know/hear – the opposite works (or at least appear to work) in IT world.

It is "power of talking in business language, very few challengers for the claims made about automation and excellent support by tool vendors" that makes GUI automation popular and tempting to attempt. Even if you fail – you can blame it on almost anyone or anything around – test cases, tool, automation folks, application changes, changing requirements, test data, operating system patches and so on.

To be continued in Part II

Eurostar 2009 – A Trip Report

I am just back from EURO STAR at Stockholm. As a ritual, and like an obedient delegate and speaker, I am narrating the experience. It was my first visit to Sweden, home for Alfred Nobel. The conference started with a bang – welcome speech by Dot Graham, the program chair for 2009 edition of biggest European software testing event. After her being first program chair at 17 years ago, she still appear to breathe same energy of someone who is attending her first testing conference.

Lee Copeland, the veteran of testing conferences in the US, who followed Dot as a key note, was at his usual calm and composed best. He spoke on next few big things in testing- a good list to watch out for. His talk also contained slides on good books for the testers to read. Probably it has become a habit for him to promote his book on "Test design" in each of this talk … (I have heard 3-4 of them). He might call it as "Shameless plug" or "No… I didn't mean it include it here", I think he should stop doing it as the book sells on its own merit. Thereafter there were parallel tracks and everyone had a choice to go to the talk that they wanted. This time I like many others, had the power of twitter to spread information about conference, with the hash tag #esconfs, I was one of those many active twitters using tweeting about the talks, and happenings at the conference. Check out all tweets related to this conference here.

As it happens in every conference, much of the action was happening outside the track sessions, people meeting, exchanging cards, and fiercely debating on topics that they hold to their hearts. There I was, roaming around trying to see which group to join. Eurostar Test Lab was in interesting addition to this year's conference, run by James Lindsey and Bart Knaack - was a big distraction (constantly pulling passionate testers towards it and away from the track sessions). I spent few hours of testing and spent some good time with James Lindsey one of my favorite exploratory testing proponent. I logged few bugs too. It was a good experience. James reminded me to explore and find out when I had a question about a feature of the application that I am testing. Here is a report from Michael Bolton on the lab

Michael Bolton was an attraction, true to his image of great speaker and testing wizard. I rarely saw him alone, always surrounded by few people, Michael was just keeping them engaged with his teaser questions and people kept coming to him. His talk - "Burning issues of Day" was a presentation inspired by collection of white board statements of conference attendees, presented in an immaculate MB style. He took mock at several testing folklores and argued convincingly against standardize and over structured software testing practices. One thing that was really remarkable in that talk was Michael's ease and confidence - with which he entered the podium and started the presentation. He was at his total ease. Many wannabe speakers should watch him starting a speech and finishing one. To me, Michael gave an example of "what it means to be being oneself". Good learning there.

It was great to meet/see likes of Jonathan Kohl, James Lindsey, Fiona Charles and Ray Arell. In the conference made few new friends – Johan Johansson, Tobias Fors, Daniel and others. That evening we got together with and headed for a dinner and followed by a meet up at a friend Pabalo's house. That was a chance for me and others to have a go at musical instruments. I settled at Drums, Michael and Pabalo took guitars. The concert went until 12-1 Am. I decided to call it a day as I need to catch up with some sleep before my conference talk next day.

It was a big day for me. Getting to speak at Eurostar has been my dream. My talk was about metrics and how they can be dangerous if used inappropriately. I wasn't nervous and was sure that I will speak my heart. I requested Ray Arell and Fiona to help me with few photos. Michael's and few others presence at my talk, helped. I managed to finish my 15 odd slides at one dot – 39 minutes. Few questions and it ended with applause. I was greatly relived doing my duty at the conference. I plan to write a separate post about that later.

I met this guy Daniel in the conference– he did not carry his business card hence do not know his coordinates. Daniel and I argued for hours about the nature of the software – physical or abstraction. Daniel insisted that software has a 3D physical existence. He even said that he had confirmed this with 2-3 physicists he knew. Software when reduced to its "atoms" or basic units is some sequences of 1's and 0's that are stored on a magnetic material. When we power the computer or a computer like devices, electric current passes through transistors and other electronic components and bring the software to life. Daniel argued that hardware components that held the sequence were real and physical. We could never take the argument to the conclusion. But it was a good argument.

Third day of the conference started with keynote on agile adoption at Intel and experience sharing by Ray Arell. It was a good session with lots of advice on getting agile right. I had to do lots of running between office and the conference venue (yes, I had some official meetings set-up on that day). I track-chaired a session by Aslak Hellesoy on "Cucumber". The session was good and well received. The name cucumber look puzzling to me as why "Cucumber" – Aslak clarified during the presentation that it was the name suggested to him by his fiancé/girlfriend. It is become fashion to name the tools/frameworks companies by totally unrelated names. Steve jobs success with Apple continues to inspire many to name their creations with the names that you typically would never make a connection. In the evening there usual conference rituals like panel discussions, prizes, vote of thanks and announcement of next year's track chair – John Fodeh. Zeger van Hese's paper wins the best paper award, Naomi Karten wins the best tutorial award and Michael Bolton wins best bug of conference. Michael wrote about it here.

In all, it was an excellent experience of being at Eurostar 2009 and at Stockholm. I will carry many memories from it for years to come… Thanks Eurostar!!!!

Don't forget to check out the photo gallery

Check out few reports about this conference

Female Funtestic Fanatic

Rikard Edgren's Test Eye

Star Tester issue 45

Note: Narrating a story or an event like a conference is tough, I think. I attempted to do it see if you like my story.