Programmers make Excellent Testers - Arguments and Counter Arguments

Janet Gregory’s post on Programmers as testers prompted me to do this post. She mentions in the post that “Programmers make excellent testers”. So I asked myself can I think of few reasons to support and few others reasons to refute the statement. Here I go …

Programmers make excellent testers because:

1. Programmers understand their code and their fellow programmer’s code /very/ well. Hence knowledge of the code helps them to test /it/ better. Creator knows the best about his/her creation having created it.
2. Programmers understand /better/ the technicalities of the platform (anything other than “software application under test”) on which their code runs.
3. Programmers can /write/ /better/ automation code that tests their product code. Writing automation is part of testing right? Or test driven development?
4. Programmers being closest to the code – can find and fix the bug in smallest time possible – hence can do efficient testing. First opportunity of finding bug in a code is with programmers. If there is a problem in the code – they are the ones that should know about it FIRST.
5. … I can’t think 5^th one … probably need some help...

Now, let me cross over to other side.

Programmers make not-so-good testers because:

1. Programmers are /usually/ blind to their own mistakes. Their testing is limited by cognitive bias (confirmation bias)
2. Programmers /typically/ good at “construction” work – getting spec to working code – not a key tester skill. Programmer testing is more like a cook tasting food made by him/her before serving.
3. Programmers testing is /typically/ happy path – where would they get time to do “out of the box” as testing, unusual paths, usability, security and performance related tests? (Unless explicitly called out as a part of specification). Programmers /often/ do not see big picture.
4. Programmers, all through their professional life, work to improve their coding skills – so testing is a part time work for them (which programmer would work to improve testing skills unless he/she decide to become full-time testers)
5. It is hard for programmers to /think/ like users (many types of them) – their mission of Spec2Code limits them to think in terms of code

A bonus: Typically programmers hate or avoid testing (other than writing automation –which is again a coding work) as far as they can. Many would say “testing is not cup of tea” but I need to do it as we all in a team are responsible for quality. Programmers can’t make excellent testers simply it is not their job (in all).

In sport like Cricket – there are specialist batsmen, bowlers and also all-rounders (who can do both equally well). That is not true for testing.

Someone might say Janet’s context is /Agile/ development model – well, how does that change my “for” and “against” arguments here? How far does that matter?

Update : Pete Houghton (Twitter @pete_houghton) mentioned that debugging is a form of testing - programmers do it well. To me, debugging is an act of hunting down a reported or known problem and fixing it. Thus debugging follows testing - not a form of the latter. By going through the process of debugging repeatedly, programmers gain understanding of how they (programmers) make mistakes and how to avoid them. That also helps them to think about interesting ideas about testing. That is one reason that helps them to be better testers.

BTW, working in customer support - gives probably the best experience for testers, through exposure to wide variety of usage patterns (many of them - out of scope of specifications). Unfortunately - end users do not use applications as per the specification or user guide.

Shrini

Book Review: Selenium 1.0 Testing Tools - Part 1

In a first of its kind request, few months back someone requested me to do a book review – that too a book on “Selenium”. I wanted to learn Selenium since Long time, to compare and contrast traditional GUI based automation that I primarily worked on.

That is how I picked up this book by David Burns – a senior software engineer in test at Mozilla Corporation (Not Mozilla Foundation) I am not an expert in Selenium – this review of mine as from the eyes of a learner. I have attempted to convey how someone like me trying to learn Selenium would find this book. Those who are with Agile movement and likes might find my comments rather unusual – bear with me. I am from a different world …

Where do I start from? Before I started reading the book – I did some reading on the web. I was looking for a good definition for Selenium. Adam Goucher defined it as “Selenium is, at its core, a set of tools which let you control a browser. What you do with that control is completely up to you. Some frequent fliers use it to reserve aisle seats on their flights, but the majority use it as part of their testing process. After all, the end user of your product is not going to be experiencing the product through their browser as well, not via some wire-line unit testing framework.” I think, Adam’s definition is simply the best one to get started.

I would strongly recommend anyone starting with Selenium to read its history and especially support from Google.

As it happens with all most all books on “selenium” – even this book claims that Selenium tests the application and not selenium is a test automation framework. To me “testing” and “automation” are two different but complementary things and I think Selenium as an Automation tool rather than a “testing” framework.

Here is an account of my experience with the book – chapter by chapter.

- The book starts off with IDE installation and Selenium- “Hello World” kind of test (record-playback). The chapter covers few additional items to sample tests such as adding comments, debugging tests, working with multiple windows. AJAX applications make a sudden appearance in the chapter - not sure why. I feel the section on AJAX is not well connected with rest of the chapter. The section on “Saving Tests” is very brief – could have been expanded to cover exporting or converting IDE tests in other languages (Java) and automation frameworks. A brief introduction of TDD or ATDD or kind of automation that Selenium supports would have been a good way to start a book. Explanation of what is “Selenium” lacks depth and can confuse beginner and experience a like.

- The chapter on “locators” helps the reader to learn about locators. Coverage on DOM, XPath, CSS is pretty good. I would have liked the introduction be deeper. Finding the elements that make up a webpage is the work of these locators. You can locate elements by ID, name, DOM (through java script), XPath and CSS selectors. If you read the introduction of the chapter and summary – I am still confused about this question – “When we can get the page elements through record and playback function – why one needs to bother about locators – so many of them”. I think I know the answer. But I would have expected the explanation to be part of the chapter… “Why one needs to learn about Locators”. If you don’t ask this question – you can very easily follow the chapter and do hands on exercises without any problem and learn about IDE and some tricks

- As it is the case with other chapters so far – the introduction of chapter “Pattern Matching” lacked the depth and connection. It does not seem introduce the chapter by attempting to answer “Why pattern matching? Where? How this is applicable to automation in selenium”. The chapter, as in others mentions “In this chapter we shall do …..” I would have expected the book to cover – why part of these activities. I would have liked to see “how these activities fit in overall landscape”. Usage of pattern matching in general has been explained well. Use of “exact”, “glob” and other nuances of regular expressions, is covered in detail.

I will cover in next posts remain chapters of the book … Do Check back

A Paradox of Reification – A Tool or a Fallacy?

Reification – Making some idea into a thing [Wikipedia]

Reification is a process (or is an essential element) of modeling – a process of reducing a problem involving multiple variables into single or few variables (For example say “drinking some brand of health drink makes you grow high or makes super intelligent”). If you see a complex (say meaning something that involves many interacting variables) problem made simple (reduced to a single variable problem), be sure to find “reification” hiding around. I would say it is a form of “reductionism”

Strangely enough, the life of Sales/marketing professionals, I think, involves nothing but reification. They can’t sell anything if they become aware of reification and start thinking about model vs. reality. A salesman selling insurance policy would say “Buy this policy – your health is secured – health is reified in terms of money you will get to pay for illness as and when it happens.

Politicians are another group of people who live by reification (I am not sure how many of them are aware of this term, though they might be aware of its effects). A politician is constantly required to create simple models of social life and impose them on his target electorate so that she can win votes. How else you think one can make promises to bring social equality, eradication of poverty by distributing rice at 2 Rupee per Kilogram or provide 70% or so reservations to certain class of people? It is reification @ work. Can any politician dare to think about (giving up) reification fallacy?

See the paradox in the act of reification. As a manager you need to support (and possibly create many new ones) reifications like “Continuous improvement”, “Customer focus (yes this is a reification), “SMART goals”, “Consistent Processes”, “Best practices”, “Metrics”, “objectivity”, “Predictability”, “knowledge transfer”, “business value”, “customer satisfaction”, “transformation” and so on. But when you become the target (rather a victim) of such reifications – you suddenly become aware of the problems of reification – while you may not know or identify by this name or label. How many of us and how often we disagree on appraisal ration ratings given by our managers (yes – your appraisal ratings are biggest and most influential reifications in our professional/corporate life).

Try asking a director, CTO, Head of Testing of a software organization – what is testing? He is most likely to talk about best practices, consistent process, business transformation, customer focus, continuous improvement etc. This person needs to reify ideas and abstractions so that he can do his job. He would not let complexities at the deep-down disturb him and simple picture is what he needs to work with.

Now, note that reification is not simplification. It is a process of abusing “simplification”, resistance to think beyond the world of abstractions. Thus, depending upon which side of the fence you find yourself in most of time, reification becomes a tool or a problem or fallacy.

In short the paradox is – you can use reification to work (mostly managers and people who need work with simple abstractions) as a tool. When it bites back to you as a victim (as in appraisals) – you would hate it and fight it as a type of “logical fallacy”.

Why wouldn’t Sales/Marketing folks think about reification as a fallacy? Do they?

In Pursuit of Quality, let’s Call Quality as something else

The theme of this year’s Euro STAR conference is “In Pursuit of Quality”. A great theme but an ambiguous I would say. I wonder what could have been the meaning/interpretation of the phrase “Quality” here. One might say “your own definition of quality” – in a group, it is a problematic one with the missing context.

It is interesting to note many in software profession acknowledge that the phrase “quality” has many meanings – different people at different instants of time paint/experience quality differently. Many admit that objectification of “quality” should be avoided. Yet, it is a rampant practice to portray “quality” as having some universal objective meaning that everyone agrees. Popular uses of the phrases like “Quality Software”, “Cost of Quality”, “Quality Engineering” etc., are examples of such practice. For many, “quality” is just a bucket where everything related to expectations of the users can be lumped together. It is one term that evokes a great array of human emotions from extreme pleasure to awe, from frustration to sense of achievement.

Let us imagine “quality” in terms of how Quantum Mechanics describes quantum states of subatomic particles. This “quality” is a like a subatomic particle in quantum superposition state and represents many meanings (or states) at any given context (or Einstein’s space-time). When one makes a swipe at this term and uses it with respect to a context saying “this is quality stuff” or “this quality is unaffordable” or “quality is free” – the wave function collapses and a particular meaning of quality shows up.

As Michael Bolton explains while posing a testing challenge about agile teams -

”For example, in a particular context, if we want “quality” to mean “bug prevention,” let’s say precisely that. Then let’s recognize the ways in which certain approaches towards preventing bugs might represent a threat to someone’s current interests or to their personal safety rules. If, in a particular context, we want quality to mean “problems solved for customers”, let’s say precisely that. Then let’s recognize that there are many approaches to solving problems, and that some problems might be solved by writing less software, not more. If we want “quality” to mean “many features in a product”, let’s say precisely that. Then let’s recognize how “many features” can satisfy some people—while adding complexity, development time, and expense to a product, thereby confusing and annoying other people. In other words, let’s use the word “quality” in a more careful way, which starts with deciding whether we want to use the word at all.”

That is to say if you want to say something to indicate quality – say it (without using the term quality). In this way, you can prevent misuse of the term for the better. If you want to pursue quality - say what you exactly mean so that people can understand you without confusion … Probably people will stop using phrases like “Cost of quality” or “Quality plan” or “Quality Assurance” or stop using quality with another set of abstractions like “Creating business value through quality software”

To repeat Michael “Quality isn’t a thing, but rather a complex set of relationships between products, people, and systems. We should be calling quality as something else that states what it is in a context rather than calling it with one word always.

I have a suggestion. Instead of using QUALITY – let us use words made of up characters in the word “QUALITY. For example - you might say LITYQUA of this software is poor to indicate usability or you might say TUAQLI of this software needs to be worked on to indicate security or cost. Once thing is sure to happen when you do this – people will say – what? When you explain them the meaning or interpretation of attribute that you are trying to represent is using the “jumbled” up word – they might relate it to that one thing and only that thing. Thus misusing and lumping “all” in to Quality, stops – I hope.
Try saying some other term when you want to say “quality” – difficult?

Two versions of being practical...

My boss tells me always that my ideas about testing are impractical and esoteric. I find it hard to understand why he thinks so. I believe my testing and automation ideas are as practical as one can get because I can demonstrate the practicality of the ideas by "doing" testing or automation. I suppose being practical - should be mean "demonstrable". I can show how my ideas of testing can be and are practical. Boss is not impressed.

Probably, my boss has another version of practicality. It goes something like this. Regardless of what subject matter or technicality of testing as it is practiced in real time - there is a social, political, business side to it. Many things that testing practitioner believes to be true are not possible in real world due to political and non technical aspects around testing. Under such situations - being practical means changing your testing philosophy and approach to suite the context of the testing world. Harsh word for such adaptability could be "compromise". If you are surprised or questioned instances like stakeholder seeking quick benefits of automation (something that reduces cost of testing), counting bugs, counting requirements, using bug metrics to measure testers, expecting testers to find all bugs, doing 100% testing - you probably are practical in another sense.

Living with many irrational notions about testing and trying to do what is possible, taking the world as it comes without challenging it or fighting the problems is the version of practicality - probably my boss believes.

Both of us could be right or wrong. How do I know?

This reminds of me of a Jerry Weinberg's saying (para phrase) - "Instead of calling something as irrational or illogical - consider it as rational or logical from another perspective or set of values".

You would like to call "testing" as an engineering pursuit? Wrong... it is becoming (it was from the beginning but now it is showing up more clearly) more social. Testing works in social systems and is affected by cultural, emotional, economical issues.

Time to start studying economics, social science to reason behavior of people in testing context. It is not rational at all times....

Shrini

What type of tester are you?

I was assisting homework of my 7 yr old daughter in her science assignment. The assignment was to collect pictures of living and non-living things and make a collage. I thought about how one could approach this work and have fun while doing it. Being a tester, it is hard not to see connection in a work like this. Here is what I thought….

Let us say two testers “A” and “B” are given this homework assignment of creating collage of living and non living things.

Tester A: Living and non living things? Just collect few pictures of animals, plants, humans - paste them as living things and collect pictures of cars, buildings, mountains, bridges and paste them as non-living things. Home work done. Test pass and on to next thing…

Tester B: While thinking about traditional ways of looking at living and non living things, this tester also questions his own understanding and meaning of “living” and “non living”. He looks around, reads, discusses with colleagues about these things. He, then ends up with a broader list and accompanying narration.

What is the difference between these ways of approaching a task at hand? Tester A is more focused on “completing” task with minimal mental engagement about the settings of the task. He takes “widely accepted” meanings and understanding of the things and optimizes the details of the task. Tester B, starts with questioning the premise of the task, meaning and understanding of the things required to complete the task. He seeks a broader understanding of task elements and attempts to question biases, prejudices of his own and others around. Though on the face of it, it might appear that Tester B will take longer time to complete the task (homework), through practice, he might able to finish the task nearly at the same time while understanding of broader aspects of the task and clarifying biases and assumptions.

Today’s testing industry promotes type A testing mentality. Type A testers eventually will become “commodity” (with price tag 20 Dollars per hour – can execute 30 test cases per hour etc). I am not sure that is a good thing or not. If you are a manager or someone who manages testing as business – you would probably have (or would like to have) more of Type A testers as such people are plenty. Simply, take truck load of freshers from college, run them through 15 days of training on testing vocabulary, make them memorize some domain stuff (banking, insurance, telecom etc) , teach QTP – there you go an army of (Type A) testers ready for deployment (hence billing).

If you are a tester looking at long term fulfilling career – you would want to take path B – practice to do things differently, challenging status-quo, thinking broadly about the problem and its context etc. To become “Type B” tester you need practice – practice of doing testing (weekend testing is an excellent platform or training ground). I am afraid there is no shortcut here – no 15 day testing courses to get there.

There is a catch here… Type B testers are mostly rebels and free thinkers and often want break status quo, create new paths, explore new territories. Much of our industry is not very nice with such folks. Very few can tolerate “ever questioning”, skeptical and rebellious tester in the team. In a control freak, military kind of set up where key focus of the job is to “follow instructions” or “getting job done” – Type B testers get frustrated. Some get converted into Type A or leave the organization. Sad situation but true in many cases. I hope that one day either rebel testers will learn to live harmoniously in a hostile, confirmatory environment or industry recognizes the value of these “problematic” folks and make space for them.

“Testers are like lighthouses at sea shores, headlights of an automobile” says James Bach (paraphrase). Good testers, apart from completing the task given to them, engage in parallel about “meta” things for better and deeper learning. That is a good thing (can I say for testers not for folks who manage testing as a business?).

Did someone say “Type C” tester?

What does that mean?

The other day I visited a science exhibition at my daughter’s school. As I was passing through an array of models and demonstrations, I could see every student was enthusiastically trying to draw every visitor's attention to his/her piece of work. Everything sounded good except one thing - the students were seen literally reading some sort of script to explain their work. Some smart guys memorized the stuff well so they did not require any hand written (mostly) note to assist them delivering the script but many did keep one by their side. It was hard to find someone who “knows” what he/she is trying to explain rather than forcing the stuff through using pale dialogue delivery.

There at the corner, a bright spectacles boy waved his hand calling me to see his work. I walked up to him. He was demonstrating “law of conservation of momentum” through a model that roughly looked like the one above.

After a customary greeting, he read through his memorized statements about law and looked at me with hopeful eyes asking if I understood anything out of it.

I asked him “what does this law tell you”.

He promptly, this time with more energy and desperately attempting to sound confident, repeated the law (the statements) like a tape recorder.

What do you understand from the term “momentum”, I asked him.

He was totally unprepared for the question (missing in his notes - missing test case?)

He quickly recovered gently tapping his spectacles said “It is simple sir - it is mass times the velocity”

I know that is a mathematical formula for momentum.

Student finally found something that I agree with him and said “Yes. you are right sir”.

At that instant, I remember my hero Richard P Feynman - the (most) curious character and asked him “But what does that mean? Mass times velocity?”

This time, student was not expecting that question at all.

I moved to another demonstration where they were showing double helix model of DNA and the girl explaining the model, gave a 5 minute non stop explanation of what DNA is and what genes are.

Again, some question “What does that mean” to the explanation “genes carry genetic code so that characteristics are passed on from species to species” got everything to a grinding halt.

I became pretty notorious from that day in among my daughter's friends about asking such silly questions about things that are taken for granted for the students. My daughter later told me that her friends were impressed with my “latest” knowledge of science and my questions.

Later, over the week end, one the students (harassed by me in the exhibition) paid a visit to my home apparently (I thought) to take revenge on me. He, incidentally was my daughters classmate and asked me “what do I do for living”. I said to him “I am a software tester”. While he did not understand that very much, he wanted to know how learn things. I explained him about few things that I learned of late as a student of software testing (than when I really a student to learn science). He appeared to have understood some of my approaches.

When he was about to leave, I gave this book “ Surely You're joking Mr Feynman” and told him - "learn science like this guy". I was happy that I introduced Feynman to a student hoping the “Feynmanism” spreads in the school.

I wish a very happy and prosperous New year to you all

Shrini

Fish baking story ...

This was a story that came to my mail box.

A little girl was watching her mother prepare a fish for dinner. Her mother cut the head and tail off the fish and then placed it into a baking pan. The little girl asked her mother why she cut the Shead and tail off the fish. Her mother thought for a while and then said, "I've always done it that way - that's how grandma did it." The girl was not satisfied with the answer, and went to visit her grandma to find out why she cut the head and tail off the fish before baking it. Grandma thought for a while and replied, "I don't know. My mother always did it that way." So the little girl and the grandma went to visit great grandma to find ask if she knew the answer. Great grandma thought for a while and said, “Because my Grandma told once that the baking pan was too small to fit in the whole fish”.

In most cases we fail to challenge belief systems and assumptions.

This is true about many ritual centric processes that we are asked to do in testing today. If you dig deeper, you would be surprised like this little girl. There are no real reasons to do certain things. Typical answers might be “my boss wants it that way”, “this is how our clients want” or “this is how I have done it always” or “this is in that book” or “this is how the consultant who we hired told us” or “this is how it always appear to work”.

One trademark quality of a skilled tester is skepticism and ability to think beyond rituals – ask “why I am doing what I am doing”.

Are you asking this question frequently in your job as tester?

Shrini

How can Mathematics be the Language of Nature?

I am reading this wonderful book about Physics (and history and philosophy of Physics) "Tao of Physics" by my all-time favorite Physicist and author Dr. Fritjof Capra, a contemporary of Werner Heisenberg. Incidentally, Dr. Capra's other famous book "Turning point" has really introduced me to the concept "systems thinking" while I struggled to grasp the subject by reading another influential legend and Guru of software (testing) Jerry Weinberg. James Bach and Michael Bolton introduced me to the subject of "systems thinking" as science of complexity.

Well... coming to the point, this post is not about system thinking but about a paragraph from the book "Tao of Physics". Here goes the text ...

"... The words of our language are thus not clearly defined... Science, on the other hand aims for clear definitions and unambiguous connections. Therefore, it abstracts the language further by limiting the meaning of its words and by standardizing its structure in accordance with the rules of logic. The ultimate abstraction takes place in mathematics where words are replaced by symbols... In this way scientists can condense information into one equation, into one single line of symbols. The view that mathematics is nothing but extremely abstracted and compressed language does not go unchallenged...mathematics not just a language to describe nature but is inherent in nature itself. Originator of this belief was Pythagoras, who made famous statement - "All things are numbers" …

This puzzles me ... considering(or conceding) that mathematics is an abstract language used by scientists to represent theories, models and scientific thought in a precise and unambiguous way, how can such an abstract language be suitable to describe nature (even in parts) ?

Nature is live, rich and multi dimensional where as mathematics is at other end of the spectrum - abstract, precise and single dimensional. Did Pythagoras and for that matter Galileo make mistake by saying "book of nature is written in the language of mathematics". I am certain that mathematics does not be to be interpreted where as nature and its manifestations need to be. Am I making sense?

Nature of Problem(s), we testers (should) investigate (or solve)

I am puzzled about the nature of problems that software testers, today are juggling with or should be losing their sleep over - more than the problems themselves. As humans, we love simplicity (not sure about about other species - not sure if there is any way to check if they love simplicity too). In every walk of life, we seek and embrace simplicity. Many experts tell the stories about how they have solved mysteries of the world through and give us the tools, techniques to conquer/dominate the world around us. We now have quantum computers even though there is barely anyone who has fully understood Quantum theory. We can make machines, apply our partial knowledge but make a mistake of claiming glory of scientific knowledge saying “this theory is true as I have experimentally verified it and there are practical applications”. The mistake that we make is about “completeness of knowledge” – making a machine based on a theory does not amount to complete understanding of the theory.


Alfred North Whitehead, the famous English mathematician and Philosopher warned us “Seek simplicity and learn to distrust it”. Simplicity is so tempting that it fools our minds and thinking and makes us glued to it. It appears to me that we have not learnt to distrust simplicity at all – worst many even does not know that we should “doubt” it. There is “banana” principle, attributed to Jerry Weinberg. For many things like simplicity, we have only learnt how to start but have horrible lost the trick “when to stop”.

Many in software world believe that software testers are expected to answer questions that, simply lead to “pass” or “fail”, “works” or “does not work”. These questions are like asking “is 4+4 =8? Is this an isosceles triangle? Is the temperature of this body 86 degree centigrade? What is National Stock market Index today?” Engineering, mathematical and scientific questions right?


In our software profession, we love to pose “engineering/scientific” questions that lead to binary answers. After all we are software ENGINEERS and software is ENGINEERED. Hence I started this post with simplicity and our obsession with simple models of complex things. What is the true nature of questions/problems that software testers investigate and solve? Let me make a bold conjecture here. I think they are similar to the questions that we pose in the realm of social sciences. For example, questions like “Is homosexuality linked to or mainly due to some mental illness?” or “Should burqa be banned in public places” or “Is there a God”. These questions are asked in a broad social setup and are linked various material practices of the people in the society, about their culture, history etc. Much in a similar way, software operates in a social context and people are mainly concerned about how software alters their day-today work. The nature of questions/problems requires non-binary, qualitative answers and solutions. The questions about quality are based on emotional, subjective feelings (there is no binary 0 or 1 about software quality).

As Michael Bolton says “decisions about quality are always emotional and political”. How can there be binary mathematics in emotions and politics? That is the point these software engineers and process people miss and insist on quantified definitions of quality and other abstract entities associated with Software. It appears to me that they are constantly conditioned by the business environment to believe so. Coming back to simplicity, software engineers (process/CMMI/Six Sigma) people definitely think that the problems that software testers/engineers solve (investigate- actually) are purely engineering/mathematical in nature. Hence they believe in absolute answers, causality, objectivity and certainty.


I think they are wrong !!!! When do we learn when to stop saying “banananananana..?” While you are reading this and interested in debating about “nature of testing problems”, many budding software testers are happily busy in churning out hundreds and thousands of test cases and blissfully shouting “pass” or “Fail”. Well, you might say do they have options … I am sure they do.

Shrini

A movement called weekend testing – What it can do to you and your testing career?

An extremely contiguous phenomenon is sweeping in some circles, communities of software testing; it is called Weekend testing (twitter @weekendtesting). Presumably, born out of few young minds (Ajay Balamurugadas, Parimala Shankariah, Sharath Byregowda, Manoj Nair ) and influenced by context driven software testing philosophy. In a short span of time, it has traveled from Bangalore (the home of Weekend testing) to Europe and the US. So it is globalized movement. As testimony of its popularity and the value, who's-who in global testing community have acknowledged it

It is hard for anyone who is a passionate/skilled software tester not to be associated with it. Once you are in, you keep going and loop in others. Such is the power of the concept and the way weekend testing works. I would probably equate (not mathematical sense) weekend testing to open source software development. What drives these movements? Passion, Desire to learn, Demonstrate skills in the real time "On Demand". The last point is something that is more apparent in weekend testing movement.

What makes weekend testing - a movement or a revolution in its infancy?

The objective:
Practice testing (or software testing) in real time with like minded (I wish if some dis-like minded folks join it – we need fight confirmation bias always as testers. Being vigilant about fallacies of "you will see what you want to see or you will fall in love with that which confirms what you know").

This objective is very powerful and has enormous potential to grow big. Keywords here are "practice" and "real time". Now you can see testers saying "I was practicing testing" just like a musician doing is "Riaz" or the sportsman sweating it out on the ground. A manager would call few of his low performing testers and say "Go and get some practice of testing in week end testing sessions". While his testers are training, the manager can check their progress by viewing the session transcripts and related blogs posted elsewhere. Transparency to the core!!!

The structure

Here is a generic format of how a weekend testing session is arranged – more detailed description can be found here

A group of testers sign-on to Skype or Google talk and get started with their testing at a specified time as announced on the weekend testing website. One of the testers assumes the role of facilitator giving the mission for the session and moderates the whole thing. Mission of the session is itself tested first – meaning it is scrutinized for possible ambiguities and a refined one is adopted. Testing starts bugs, issues are logged in an open source bug database and followed by a debrief session. The session transcript is posted with results for everyone to see and learn further. It is almost free for those that have computers and internet connection. Weekend testing thrives on usage of open source/free tools. It is more or less self-driven – all that was required to set up an initial structure for testers to get started and posting the results and transcripts of the sessions.

It is a terrific place to learn With so many willing, skilled and available testers and coaches around what you are learn is nearly unlimited. With a wide variety of products to test, variety of testing missions to try out – there is something new to learn in every session. With people across the globe participating, you get access to an interesting cross section of skills, cultural legacies to learn from. You can improve your communication, writing skills in a friendly yet mildly critical environment. People here tend to me cooperative and more interested in seeing everyone succeed. It is the genuine interest in other's growth and improvement that makes weekend testing a unique platform for greenhorns to gaze around.

It promotes "community" certification/acknowledge - not given by any "standards" like bodies. Needless to say the people associated with the movement are known for their "critical thinking" and would go to any extent to be "open for scrutiny/challenge any time anywhere". That makes this movement strong and really unbeatable.

It makes testing demonstrable in its deepest details possible. For long, people influenced by context driven testing have struggled to explain the power of exploratory testing. They have been trying to argue for skill in testing as opposed to so called best practices and process driven testing. And there "test cases" the ubiquitous building blocks of testing and the questions that were often asked "how can you do testing without test cases, How can you repeat testing? How will client accept that we tested?" Now we have a proof in a public movement and "open for scrutiny anyone anytime" – You can literally see testing happening in front of your eyes. What more evidence you need?

The People

The founding members of this movement are, at the core, very dedicated, passionate software testers whose sole distinguishing qualification is their appetite for learning and become skilled software testers. That drives them to do things that they do. They are constantly building their reputation through demonstrating their skill and learning constantly. So, hanging around with these people can be very dangerous. They will make you like them. The weekend community is very aptly mentored, coached and supported by people like Michael Bolton (no… not the singer or the "officespace" character), James Bach, Cem Kaner, Pradeep Soundararajan, Matt Heusser, Jerry Weinberg, just to name a few.

We (notice, I am using this "we" after a long hesitation as I have not contributed enough to include myself in this movement) are thankful to these leading lights for encouraging the movement and advertizing it.

The Future of weekend testing – The way I see it.

Few thoughts …

• There will be many chapters of weekend testing all over the world. There will be weekend testing conferences to the scale of STAR conferences in the US.
  
• Big universities teaching software engineering will recognize this movement and will have representatives from them
  
• Commercial product companies will engage weekend testing folks to test their products. With the support of crowd sourcing companies like utest, week end testing will gain authenticity and grow.
  
• Weekend testing will then get into weekdays as people want more of it – we then have to look for another name to call and identify ourselves … But that is an interesting problem to solve
  

I am looking forward to participate and contribute this movement …

Update (10th March 2010)

Parimala Shankariah has brought my notice that Santhosh Tuppad, yet another promising testing star in the indian software testing horizon, who was recently featured on utest - has been a very contributing member of this movement. Thanks Parimala for point this out ....

Santosh ... nice to know about your contributions.. Keep going... Look forward to participate in sessions with you.

Shrini

Man and the machine ....

Can a software be produced and maintained using a machines in a factory … like Google does?

Notice, with their machinery and factory setup, they have been able to cut down the testing cycle to 4 minutes... Sounds unbelievable .. I think it is ..

Google produces Innovation machines and IT vendors set up factories… So back to Factories and start oiling machines. Probably we should change IT designations to map roles on assembly line shop floor … - foreman, supervisor (this is there already in some sense), operator, inventory manager, assembler…. As few expect, IT work will eventually be a factory job. For one, my wife and kids would be happy. I can then work in shifts and come back home at sharp, say 5:30.

It is interesting to watch what Google is doing ….though their innovation factory recently had an outage…. With “Buzz” having “bugs” related to privacy...

You know what … as a tester, bugs make me happy… that is an eccentric , one sided view …especially with those who boast “machines and factories” of software. After all we are very far from the day when truly software will be produced in machines, requirements will be read from humans through bar-code type readers, we would be able recall software as cars (instead of sending a patch) and there would software garages.

Dreaming …!!! Someone woke me up... It is a brand new Monday….

Shrini

I’m blocked… but why?

"There is no such thing as writer's block for writers whose standards are low enough." American Poet William Stafford

I am blocked from doing any interesting writing for over 2 month or so. And this post is all about the condition of being blocked in writing.

As paradoxical as it sounds, I have not been able to write anything to my satisfaction so that I can publish. In the world of writing, they say it is a condition called "writer's block". It's a terrible feeling for me as block keeps haunting on every attempt to complete a half finished work or a new one. Here I am sitting with over a dozen of unfinished posts, like a kid confused about which toy to pick and play. It is funny that not long ago, I was chatting with Parimala Shankaraiah, a (or the?) curious tester about this condition called "writer's block". I suggested to her that when blocked, try writing about it. That is what I am exactly doing now. To illustrate the influence of WB on me, I am planning to keep this post small and focused on the topic and close it as soon as possible.

For non starters, WB is a temporary mental condition of extreme self censorship leading to rejection of semi finished parts of writing as this reference suggests. Keyword here is self imposed censorship. Those getting stuck in WB, can take conform in William Stafford's statement consoling themselves with pat on their backs for their high standards of writing.

Why I am blocked on writing?

While I am not short of ideas to write, when I pick up something to write after writing few paragraphs, I start getting a strange feeling that I am repeating or the plot of writing is hypnotically drawn into a predetermined destiny. That is where I stop writing. Not sure what is happening.

WB, like inattentional blindness appears to occur when you are ultra/super focused on a topic so that you literally stop thinking. I am not sure, my state of WB has anything to do my focus (or lack of it) on writing. As defocusing tactic, I switched to twitter and tried my hand there sufficiently enough to get an attention from none other than Jon Bach. That still leaves me high and dry and I am back to WB. Another thing that I suggested to self – stop writing but speak (do pod casts) or draw/paint (cartoons etc). I need to try these channels to see if they help. I think what happens in WB is one (of several) channels of expression of thought gets blocked while keeping others intact.

One thing is sure; I am under pressure of not being able to write even though I have many "good" ideas to write. Some say WB leads to lack of motivation or happens due to lack of motivation. I am pretty sure that is not the case with me. Anxiety, fear of being ignored or failure certainly is back of my mind.

Aiming for High standards? High degree of censorship? Yes, that appears to be the case. I reject lots of writing (pile of trashed pieces of writing) as they don't qualify my own criteria to be good enough to be posted.

I don't agree to generic prescriptions like Lack of preparation, reading, and picking up boring topic leads to WB. My experience of WB is that I am blocked on those topics I deeply interested in and dying to writing something significant, lack of preparation/reading might be an issue in some cases but the in case of the ideas that are pretty original, I often avoid reading to get my flow of my "unbiased" ideas.

Well, now I am suddenly realized that I am supposed to suffering from WB…let me abruptly end this post and start mulling over the cures for WB.

Oh!!! What a relief. This post comes to an end. Have I overcome writers block? Over to you…dear reader!!!

Note: I am surprised to note that there are other types of writer's blocks such as this and this. Here the word block is intended as "collection" or "store" of useful writer tools. Same name different connotation.

Why GUI Test Automation is Popular and Tempting? Part I - Business context

It is rare to someone on testing in IT world not be aware of "QTP, WinRunner, Rational Robot etc ". For the most in IT world, the word automation is synonymous with GUI automation. Perhaps some even might ask - "Are there any other forms of automation other GUI automation". Lots of how IT world sees automation has to do with how they see testing itself. You might have heard this "celebrated" statement - "Regression testing is more suitable to do automation (read GUI) so that we don't have to have our IT staff do that unnecessary testing manually".

To my surprise, having been associated with IT for many years now, people here still believe in perfect requirement, perfect test case and hence "doing it right first time". In contrast to this, software product world (likes of Microsoft, Google) treats test automation and hence testing in a total different way. These two probably operate from two extreme ends of automation spectrum. While discussion how automation is perceived and realized in IT and software product world can be a separate post in itself, I propose to elaborate here my views on the popularity on GUI Automation in IT.

Testing is necessary evil and should get done quickly

This is the reason #1 that drives GUI automation in IT. Business spends on getting IT application features developed and hence often fails to see a reason why they should spend on "testing" at all. Business feels that IT should get their acts together, get aligned to business (hence IT-business alignment is often a well understood but poorly executed theme) and use the money towards delivering business value. I even have heard some business folks saying "We pay for the application features not for their testing". Hence funding testing becomes IT's problem. That is why they look for solutions to reduce testing (or spend on it). As simple as it sounds, IT pays for testing (not the business)

Testers (read business users or analysts) are hard to find and they hate regression testing

Traditionally, in IT business analysts (aka BA's) are hot commodities. They do most of what we call as "testing". With outsourcing wave sweeping IT, some of BA roles are taken by either BA's from outsourcing vendors or some junior tester working under supervision of BA through highly scripted test procedures. The problem is that these BA resources are few and are in great demand. But as software releases happen, changes come in – testing (regression testing) has to happen and BA's hate regression testing (or all of testing). So, an IT manager needs to find a way out to carry out this regression testing somehow.

How to get testing done fast and cheaper?

If you want to do something fast, give it to machine or a computer program - goes a typical approach of an IT manager. So automation could be the answer. If you position automation as an aid to reduce the cycle time to market chances of you getting funding is very high. Surprisingly this claim is so powerful that when made often goes unchallenged. The magic word "reduction of cycle time" (that too without any qualification) is tempting for many that it will make them to completely overlook bottlenecks in development (programming), bug fixing, investigation and any other non-testing activities. Thus speaking in terms of business language is so important in proliferation and apparent success of GUI automation in IT.

With GUI automation, you can easily make a business case for automation.

To me, the terms like test cases, regression testing, test cycle time etc are the ones that business stakeholders appear to understand very well. As compared to unit tests and other forms of non GUI automation, you can make your case for investment in GUI automation to business stakeholders. Let us say you have 5000 test cases for a regression testing to cover that takes 5 person days to complete, automate 50% of them you can straight way knock off 50% of your testing cycle time reduced. This claim due to its simple math and logic makes a compelling case for GUI automation or simply "automation". What else can be more appealing than investing once in automation (for the sake of making the business case and further forever – down play the maintenance costs) and saving on a percentage proportional amount of automation achieved for 100's of cycles in the future?

Tool vendors help you by talking in business language

Enter tool vendors – with the fancy looking charts, scary quotes like "more than 60% of project cost goes to testing, what are you doing to reduce it" and likes Gartner's and Forrester's to support the claims with numbers, life for an IT manager in charge of testing has never been so easy. Just buy a tool, engage an outsourcing vendor to do GUI automation – all your testing worries are over in one go. Dreaming never ends here. Take a look at any brochures, websites, commercial literatures – they are full of business terms like ROI, Cost of testing, business impact, and Time to market and so on - making a perfect connect with those that matter in making decisions about spending.

Now try doing this (or these) with non GUI tests such as API tests or xUnit framework based unit tests – what will business say? What will IT managers say – "well, developers got to do that testing anyway"? So there is no apparent business case to make so sell automation. Writing unit tests will not speedup your testing cycle, will not reduce time to market, might reduce some portion of that "boring" regression testing – but. So you have no financially oriented incentive to do any automation that is non GUI. Since GUI automation paradigm as so much hardened into dogma – very few challenge it.

If you are the one working in a software product company – you would say "do less of GUI automation, they are very brittle and costly to maintain". You would be surprise to know/hear – the opposite works (or at least appear to work) in IT world.

It is "power of talking in business language, very few challengers for the claims made about automation and excellent support by tool vendors" that makes GUI automation popular and tempting to attempt. Even if you fail – you can blame it on almost anyone or anything around – test cases, tool, automation folks, application changes, changing requirements, test data, operating system patches and so on.

To be continued in Part II

Eurostar 2009 – A Trip Report

I am just back from EURO STAR at Stockholm. As a ritual, and like an obedient delegate and speaker, I am narrating the experience. It was my first visit to Sweden, home for Alfred Nobel. The conference started with a bang – welcome speech by Dot Graham, the program chair for 2009 edition of biggest European software testing event. After her being first program chair at 17 years ago, she still appear to breathe same energy of someone who is attending her first testing conference.

Lee Copeland, the veteran of testing conferences in the US, who followed Dot as a key note, was at his usual calm and composed best. He spoke on next few big things in testing- a good list to watch out for. His talk also contained slides on good books for the testers to read. Probably it has become a habit for him to promote his book on "Test design" in each of this talk … (I have heard 3-4 of them). He might call it as "Shameless plug" or "No… I didn't mean it include it here", I think he should stop doing it as the book sells on its own merit. Thereafter there were parallel tracks and everyone had a choice to go to the talk that they wanted. This time I like many others, had the power of twitter to spread information about conference, with the hash tag #esconfs, I was one of those many active twitters using tweeting about the talks, and happenings at the conference. Check out all tweets related to this conference here.

As it happens in every conference, much of the action was happening outside the track sessions, people meeting, exchanging cards, and fiercely debating on topics that they hold to their hearts. There I was, roaming around trying to see which group to join. Eurostar Test Lab was in interesting addition to this year's conference, run by James Lindsey and Bart Knaack - was a big distraction (constantly pulling passionate testers towards it and away from the track sessions). I spent few hours of testing and spent some good time with James Lindsey one of my favorite exploratory testing proponent. I logged few bugs too. It was a good experience. James reminded me to explore and find out when I had a question about a feature of the application that I am testing. Here is a report from Michael Bolton on the lab

Michael Bolton was an attraction, true to his image of great speaker and testing wizard. I rarely saw him alone, always surrounded by few people, Michael was just keeping them engaged with his teaser questions and people kept coming to him. His talk - "Burning issues of Day" was a presentation inspired by collection of white board statements of conference attendees, presented in an immaculate MB style. He took mock at several testing folklores and argued convincingly against standardize and over structured software testing practices. One thing that was really remarkable in that talk was Michael's ease and confidence - with which he entered the podium and started the presentation. He was at his total ease. Many wannabe speakers should watch him starting a speech and finishing one. To me, Michael gave an example of "what it means to be being oneself". Good learning there.

It was great to meet/see likes of Jonathan Kohl, James Lindsey, Fiona Charles and Ray Arell. In the conference made few new friends – Johan Johansson, Tobias Fors, Daniel and others. That evening we got together with and headed for a dinner and followed by a meet up at a friend Pabalo's house. That was a chance for me and others to have a go at musical instruments. I settled at Drums, Michael and Pabalo took guitars. The concert went until 12-1 Am. I decided to call it a day as I need to catch up with some sleep before my conference talk next day.

It was a big day for me. Getting to speak at Eurostar has been my dream. My talk was about metrics and how they can be dangerous if used inappropriately. I wasn't nervous and was sure that I will speak my heart. I requested Ray Arell and Fiona to help me with few photos. Michael's and few others presence at my talk, helped. I managed to finish my 15 odd slides at one dot – 39 minutes. Few questions and it ended with applause. I was greatly relived doing my duty at the conference. I plan to write a separate post about that later.

I met this guy Daniel in the conference– he did not carry his business card hence do not know his coordinates. Daniel and I argued for hours about the nature of the software – physical or abstraction. Daniel insisted that software has a 3D physical existence. He even said that he had confirmed this with 2-3 physicists he knew. Software when reduced to its "atoms" or basic units is some sequences of 1's and 0's that are stored on a magnetic material. When we power the computer or a computer like devices, electric current passes through transistors and other electronic components and bring the software to life. Daniel argued that hardware components that held the sequence were real and physical. We could never take the argument to the conclusion. But it was a good argument.

Third day of the conference started with keynote on agile adoption at Intel and experience sharing by Ray Arell. It was a good session with lots of advice on getting agile right. I had to do lots of running between office and the conference venue (yes, I had some official meetings set-up on that day). I track-chaired a session by Aslak Hellesoy on "Cucumber". The session was good and well received. The name cucumber look puzzling to me as why "Cucumber" – Aslak clarified during the presentation that it was the name suggested to him by his fiancé/girlfriend. It is become fashion to name the tools/frameworks companies by totally unrelated names. Steve jobs success with Apple continues to inspire many to name their creations with the names that you typically would never make a connection. In the evening there usual conference rituals like panel discussions, prizes, vote of thanks and announcement of next year's track chair – John Fodeh. Zeger van Hese's paper wins the best paper award, Naomi Karten wins the best tutorial award and Michael Bolton wins best bug of conference. Michael wrote about it here.

In all, it was an excellent experience of being at Eurostar 2009 and at Stockholm. I will carry many memories from it for years to come… Thanks Eurostar!!!!

Don't forget to check out the photo gallery

Check out few reports about this conference

Female Funtestic Fanatic

Rikard Edgren's Test Eye

Star Tester issue 45

Note: Narrating a story or an event like a conference is tough, I think. I attempted to do it see if you like my story.