Programmers make Excellent Testers - Arguments and Counter Arguments

Janet Gregory’s post on Programmers as testers prompted me to do this post. She mentions in the post that “Programmers make excellent testers”. So I asked myself can I think of few reasons to support and few others reasons to refute the statement. Here I go …

Programmers make excellent testers because:

1. Programmers understand their code and their fellow programmer’s code /very/ well. Hence knowledge of the code helps them to test /it/ better. Creator knows the best about his/her creation having created it.
2. Programmers understand /better/ the technicalities of the platform (anything other than “software application under test”) on which their code runs.
3. Programmers can /write/ /better/ automation code that tests their product code. Writing automation is part of testing right? Or test driven development?
4. Programmers being closest to the code – can find and fix the bug in smallest time possible – hence can do efficient testing. First opportunity of finding bug in a code is with programmers. If there is a problem in the code – they are the ones that should know about it FIRST.
5. … I can’t think 5^th one … probably need some help...

Now, let me cross over to other side.

Programmers make not-so-good testers because:

1. Programmers are /usually/ blind to their own mistakes. Their testing is limited by cognitive bias (confirmation bias)
2. Programmers /typically/ good at “construction” work – getting spec to working code – not a key tester skill. Programmer testing is more like a cook tasting food made by him/her before serving.
3. Programmers testing is /typically/ happy path – where would they get time to do “out of the box” as testing, unusual paths, usability, security and performance related tests? (Unless explicitly called out as a part of specification). Programmers /often/ do not see big picture.
4. Programmers, all through their professional life, work to improve their coding skills – so testing is a part time work for them (which programmer would work to improve testing skills unless he/she decide to become full-time testers)
5. It is hard for programmers to /think/ like users (many types of them) – their mission of Spec2Code limits them to think in terms of code

A bonus: Typically programmers hate or avoid testing (other than writing automation –which is again a coding work) as far as they can. Many would say “testing is not cup of tea” but I need to do it as we all in a team are responsible for quality. Programmers can’t make excellent testers simply it is not their job (in all).

In sport like Cricket – there are specialist batsmen, bowlers and also all-rounders (who can do both equally well). That is not true for testing.

Someone might say Janet’s context is /Agile/ development model – well, how does that change my “for” and “against” arguments here? How far does that matter?

Update : Pete Houghton (Twitter @pete_houghton) mentioned that debugging is a form of testing - programmers do it well. To me, debugging is an act of hunting down a reported or known problem and fixing it. Thus debugging follows testing - not a form of the latter. By going through the process of debugging repeatedly, programmers gain understanding of how they (programmers) make mistakes and how to avoid them. That also helps them to think about interesting ideas about testing. That is one reason that helps them to be better testers.

BTW, working in customer support - gives probably the best experience for testers, through exposure to wide variety of usage patterns (many of them - out of scope of specifications). Unfortunately - end users do not use applications as per the specification or user guide.

Shrini

Book Review: Selenium 1.0 Testing Tools - Part 1

In a first of its kind request, few months back someone requested me to do a book review – that too a book on “Selenium”. I wanted to learn Selenium since Long time, to compare and contrast traditional GUI based automation that I primarily worked on.

That is how I picked up this book by David Burns – a senior software engineer in test at Mozilla Corporation (Not Mozilla Foundation) I am not an expert in Selenium – this review of mine as from the eyes of a learner. I have attempted to convey how someone like me trying to learn Selenium would find this book. Those who are with Agile movement and likes might find my comments rather unusual – bear with me. I am from a different world …

Where do I start from? Before I started reading the book – I did some reading on the web. I was looking for a good definition for Selenium. Adam Goucher defined it as “Selenium is, at its core, a set of tools which let you control a browser. What you do with that control is completely up to you. Some frequent fliers use it to reserve aisle seats on their flights, but the majority use it as part of their testing process. After all, the end user of your product is not going to be experiencing the product through their browser as well, not via some wire-line unit testing framework.” I think, Adam’s definition is simply the best one to get started.

I would strongly recommend anyone starting with Selenium to read its history and especially support from Google.

As it happens with all most all books on “selenium” – even this book claims that Selenium tests the application and not selenium is a test automation framework. To me “testing” and “automation” are two different but complementary things and I think Selenium as an Automation tool rather than a “testing” framework.

Here is an account of my experience with the book – chapter by chapter.

- The book starts off with IDE installation and Selenium- “Hello World” kind of test (record-playback). The chapter covers few additional items to sample tests such as adding comments, debugging tests, working with multiple windows. AJAX applications make a sudden appearance in the chapter - not sure why. I feel the section on AJAX is not well connected with rest of the chapter. The section on “Saving Tests” is very brief – could have been expanded to cover exporting or converting IDE tests in other languages (Java) and automation frameworks. A brief introduction of TDD or ATDD or kind of automation that Selenium supports would have been a good way to start a book. Explanation of what is “Selenium” lacks depth and can confuse beginner and experience a like.

- The chapter on “locators” helps the reader to learn about locators. Coverage on DOM, XPath, CSS is pretty good. I would have liked the introduction be deeper. Finding the elements that make up a webpage is the work of these locators. You can locate elements by ID, name, DOM (through java script), XPath and CSS selectors. If you read the introduction of the chapter and summary – I am still confused about this question – “When we can get the page elements through record and playback function – why one needs to bother about locators – so many of them”. I think I know the answer. But I would have expected the explanation to be part of the chapter… “Why one needs to learn about Locators”. If you don’t ask this question – you can very easily follow the chapter and do hands on exercises without any problem and learn about IDE and some tricks

- As it is the case with other chapters so far – the introduction of chapter “Pattern Matching” lacked the depth and connection. It does not seem introduce the chapter by attempting to answer “Why pattern matching? Where? How this is applicable to automation in selenium”. The chapter, as in others mentions “In this chapter we shall do …..” I would have expected the book to cover – why part of these activities. I would have liked to see “how these activities fit in overall landscape”. Usage of pattern matching in general has been explained well. Use of “exact”, “glob” and other nuances of regular expressions, is covered in detail.

I will cover in next posts remain chapters of the book … Do Check back

A Paradox of Reification – A Tool or a Fallacy?

Reification – Making some idea into a thing [Wikipedia]

Reification is a process (or is an essential element) of modeling – a process of reducing a problem involving multiple variables into single or few variables (For example say “drinking some brand of health drink makes you grow high or makes super intelligent”). If you see a complex (say meaning something that involves many interacting variables) problem made simple (reduced to a single variable problem), be sure to find “reification” hiding around. I would say it is a form of “reductionism”

Strangely enough, the life of Sales/marketing professionals, I think, involves nothing but reification. They can’t sell anything if they become aware of reification and start thinking about model vs. reality. A salesman selling insurance policy would say “Buy this policy – your health is secured – health is reified in terms of money you will get to pay for illness as and when it happens.

Politicians are another group of people who live by reification (I am not sure how many of them are aware of this term, though they might be aware of its effects). A politician is constantly required to create simple models of social life and impose them on his target electorate so that she can win votes. How else you think one can make promises to bring social equality, eradication of poverty by distributing rice at 2 Rupee per Kilogram or provide 70% or so reservations to certain class of people? It is reification @ work. Can any politician dare to think about (giving up) reification fallacy?

See the paradox in the act of reification. As a manager you need to support (and possibly create many new ones) reifications like “Continuous improvement”, “Customer focus (yes this is a reification), “SMART goals”, “Consistent Processes”, “Best practices”, “Metrics”, “objectivity”, “Predictability”, “knowledge transfer”, “business value”, “customer satisfaction”, “transformation” and so on. But when you become the target (rather a victim) of such reifications – you suddenly become aware of the problems of reification – while you may not know or identify by this name or label. How many of us and how often we disagree on appraisal ration ratings given by our managers (yes – your appraisal ratings are biggest and most influential reifications in our professional/corporate life).

Try asking a director, CTO, Head of Testing of a software organization – what is testing? He is most likely to talk about best practices, consistent process, business transformation, customer focus, continuous improvement etc. This person needs to reify ideas and abstractions so that he can do his job. He would not let complexities at the deep-down disturb him and simple picture is what he needs to work with.

Now, note that reification is not simplification. It is a process of abusing “simplification”, resistance to think beyond the world of abstractions. Thus, depending upon which side of the fence you find yourself in most of time, reification becomes a tool or a problem or fallacy.

In short the paradox is – you can use reification to work (mostly managers and people who need work with simple abstractions) as a tool. When it bites back to you as a victim (as in appraisals) – you would hate it and fight it as a type of “logical fallacy”.

Why wouldn’t Sales/Marketing folks think about reification as a fallacy? Do they?