All wannabe software testers out there …

An anonymous comment posted on my blog read “hi i am non IT background i want to do software testing course in india. if some give me some of the institution in india who teach very well that help u when u get job for software testing and ISTQB test exam.”

Nothing new here, I receive mails regularly asking for suggestion on how to get on a fast track for software testing and get a job. This particular post has three dangerous things that I see – which a new entrant in software testing should be aware of and avoid them.

“Software Testing course” – Let me tell you from my experience – there no such course in world that can make you a tester worth a job overnight. Any course that claims to do is a complete hoax and fraud. Shorter the course duration and taller the claims made by it – deadlier it is. Folks – be aware of such courses that claim you to get a testers job – please don’t fall into the trap. Another dangerous mix or variation here is – claim of “teaching automation or one or more world leading automation tool”. If you want to be software tester - no matter whether you are from IT or non IT background – don’t waste your money and/or time on such courses.

“ISTQB” (replace this with any popular testing certification” – while much has been written by my colleagues and many real life (not so pleasant) experiences out there – I would want to touch upon one thing. ISTQB or for that matter any testing certification WILL NOT teach you how to DO testing and how to gain expertise at it. That is their limitation. Please understand it. Considering certifications as businesses making money – teaching testing and assessing testing skill of a candidate in real time is not their cup of tea. Real certifications and exams that do subjective assessment of testing skill – are not scalable and hence certification people can’t make fast money.

ISTQB and other certifications have done one thing well – marketing. In India, many organizations and recruiters insist one or other testing certification as a must for an entry level testing job. In some others, attaining certifications is a criterion for promotion. Sad state of affairs – though. Sad – because it sets a wrong precedent. It creates wrong expectations among entrants and companies that hire these newbies. It creates a wrong image about software testing in general and at times it trivializes the craft. I have people telling me how easy was to pass a certification exam when they had no prior background or practice about testing. At the most you can get to know some terms used in testing and their fixed meaning as used by a specific group of people. Worst - in some cases they are taught as though those terms have universal meaning and acceptance.

Getting Job – this is third danger in the wannabe’s should avoid. While in some cases you might be lucky get a job using some testing course or certification (hiring process for software testing in India – lot needs to improve – but that is a different topic) – you will not survive long unless you practice real testing – get your hands dirty, feel and learn to think like tester. I can relate software testing skill to that of a musician. If you want to be an expert guitarist – what would you do? Take a 2 week course and a certification (theory) exam and claim a professional guitarist job? As taking a 2 week guitar crash course will not make you a competent – taking a software testing course will not prepare you for a professional career. Similarly learning few words and vocabulary related to music and guitar – will not help you to give performances – though knowing common words and their meanings can be helpful. This is not much different with respect to software testing – getting an ISTQB can help you to know words like “regression testing” and “severity of a software bug” – but those are mere words. Role of ISTQB ends there and real work of practicing testing starts.

In a nutshell – if you are reading this post, is someone who is trying to make it to software testing, want to get an IT job in software testing – here is what you need to keep in mind

Don’t do these things:

1. Look for or ask for software testing institute that gives a software testing course (shorter the better. One that guarantees job – the gem)
2. Taking a certification – especially when you have no clue on what is software testing.
3. Look for job on the basis of 1 and 2 above.
4. Don’t take a crash course on automation or automation tools in order to get a software testing job.
5. Take shortcuts

Do these things (few of things that personally helped me in my career) – Try them

1. Have a time horizon of 1-2 years to the minimum and a complete dedication to learn and practice software testing
2. Get a mentor – there are many willing to mentor if you show real passion and dedication. Read software testing blogs and engage in conversation. Use social media to your advantage – blogs, twitter, facebook, linkedIn. Make your presence felt as hungry, passionate new comer in software testing – let world notice you. Build reputation.
3. Practice testing - A platform I recommend is weekendtesting.

Any questions?

Shrini

How IT deals with Test Automation ...

Here is a short post on how test automation is dealt in IT/IT services organizations.

For example : A typical test cycle:

Before Automation -- (AS IS statistics - documented)

Number of Manual test cases = 300

Time taken to execute these test cases = 10 Person days

Tester's productivity = 30 Test cases per day (derived data)

After Automation -- (Assume 150 of these test cases are automated)

How do you think test cycle would look like?

Number of test cases Automated = 150

Number of test cases requiring manual effort = 150

Time taken for 300 test cases = Time for Automated execution + Time for Manual test execution

= (how much time will 150 automated test cases will take to complete) + 5 days (with 30 test cases per day productivity)

= 1 + 5 (assuming that automated tests run at 1/5th of time of manual execution)

= 6 days

Hence a business case will be developed to justify automation investment

The business case would say - with automation, we can save 4 person days of testing effort per cycle. Depending upon how many cycles of testing are lined up - we can break even the automation investment.

Oh... What a great feeling ... This is what I see all day in and out..

Let me tell you one thing .. executives, managers love this stuff ... so neat - objectively stated in numbers, justifying in terms of monetary terms why we need automation.

But some discomfort and terrible feeling in me ... what you people think - we are missing here?

Programmers make Excellent Testers - Arguments and Counter Arguments

Janet Gregory’s post on Programmers as testers prompted me to do this post. She mentions in the post that “Programmers make excellent testers”. So I asked myself can I think of few reasons to support and few others reasons to refute the statement. Here I go …

Programmers make excellent testers because:

1. Programmers understand their code and their fellow programmer’s code /very/ well. Hence knowledge of the code helps them to test /it/ better. Creator knows the best about his/her creation having created it.
2. Programmers understand /better/ the technicalities of the platform (anything other than “software application under test”) on which their code runs.
3. Programmers can /write/ /better/ automation code that tests their product code. Writing automation is part of testing right? Or test driven development?
4. Programmers being closest to the code – can find and fix the bug in smallest time possible – hence can do efficient testing. First opportunity of finding bug in a code is with programmers. If there is a problem in the code – they are the ones that should know about it FIRST.
5. … I can’t think 5^th one … probably need some help...

Now, let me cross over to other side.

Programmers make not-so-good testers because:

1. Programmers are /usually/ blind to their own mistakes. Their testing is limited by cognitive bias (confirmation bias)
2. Programmers /typically/ good at “construction” work – getting spec to working code – not a key tester skill. Programmer testing is more like a cook tasting food made by him/her before serving.
3. Programmers testing is /typically/ happy path – where would they get time to do “out of the box” as testing, unusual paths, usability, security and performance related tests? (Unless explicitly called out as a part of specification). Programmers /often/ do not see big picture.
4. Programmers, all through their professional life, work to improve their coding skills – so testing is a part time work for them (which programmer would work to improve testing skills unless he/she decide to become full-time testers)
5. It is hard for programmers to /think/ like users (many types of them) – their mission of Spec2Code limits them to think in terms of code

A bonus: Typically programmers hate or avoid testing (other than writing automation –which is again a coding work) as far as they can. Many would say “testing is not cup of tea” but I need to do it as we all in a team are responsible for quality. Programmers can’t make excellent testers simply it is not their job (in all).

In sport like Cricket – there are specialist batsmen, bowlers and also all-rounders (who can do both equally well). That is not true for testing.

Someone might say Janet’s context is /Agile/ development model – well, how does that change my “for” and “against” arguments here? How far does that matter?

Update : Pete Houghton (Twitter @pete_houghton) mentioned that debugging is a form of testing - programmers do it well. To me, debugging is an act of hunting down a reported or known problem and fixing it. Thus debugging follows testing - not a form of the latter. By going through the process of debugging repeatedly, programmers gain understanding of how they (programmers) make mistakes and how to avoid them. That also helps them to think about interesting ideas about testing. That is one reason that helps them to be better testers.

BTW, working in customer support - gives probably the best experience for testers, through exposure to wide variety of usage patterns (many of them - out of scope of specifications). Unfortunately - end users do not use applications as per the specification or user guide.

Shrini

Book Review: Selenium 1.0 Testing Tools - Part 1

In a first of its kind request, few months back someone requested me to do a book review – that too a book on “Selenium”. I wanted to learn Selenium since Long time, to compare and contrast traditional GUI based automation that I primarily worked on.

That is how I picked up this book by David Burns – a senior software engineer in test at Mozilla Corporation (Not Mozilla Foundation) I am not an expert in Selenium – this review of mine as from the eyes of a learner. I have attempted to convey how someone like me trying to learn Selenium would find this book. Those who are with Agile movement and likes might find my comments rather unusual – bear with me. I am from a different world …

Where do I start from? Before I started reading the book – I did some reading on the web. I was looking for a good definition for Selenium. Adam Goucher defined it as “Selenium is, at its core, a set of tools which let you control a browser. What you do with that control is completely up to you. Some frequent fliers use it to reserve aisle seats on their flights, but the majority use it as part of their testing process. After all, the end user of your product is not going to be experiencing the product through their browser as well, not via some wire-line unit testing framework.” I think, Adam’s definition is simply the best one to get started.

I would strongly recommend anyone starting with Selenium to read its history and especially support from Google.

As it happens with all most all books on “selenium” – even this book claims that Selenium tests the application and not selenium is a test automation framework. To me “testing” and “automation” are two different but complementary things and I think Selenium as an Automation tool rather than a “testing” framework.

Here is an account of my experience with the book – chapter by chapter.

- The book starts off with IDE installation and Selenium- “Hello World” kind of test (record-playback). The chapter covers few additional items to sample tests such as adding comments, debugging tests, working with multiple windows. AJAX applications make a sudden appearance in the chapter - not sure why. I feel the section on AJAX is not well connected with rest of the chapter. The section on “Saving Tests” is very brief – could have been expanded to cover exporting or converting IDE tests in other languages (Java) and automation frameworks. A brief introduction of TDD or ATDD or kind of automation that Selenium supports would have been a good way to start a book. Explanation of what is “Selenium” lacks depth and can confuse beginner and experience a like.

- The chapter on “locators” helps the reader to learn about locators. Coverage on DOM, XPath, CSS is pretty good. I would have liked the introduction be deeper. Finding the elements that make up a webpage is the work of these locators. You can locate elements by ID, name, DOM (through java script), XPath and CSS selectors. If you read the introduction of the chapter and summary – I am still confused about this question – “When we can get the page elements through record and playback function – why one needs to bother about locators – so many of them”. I think I know the answer. But I would have expected the explanation to be part of the chapter… “Why one needs to learn about Locators”. If you don’t ask this question – you can very easily follow the chapter and do hands on exercises without any problem and learn about IDE and some tricks

- As it is the case with other chapters so far – the introduction of chapter “Pattern Matching” lacked the depth and connection. It does not seem introduce the chapter by attempting to answer “Why pattern matching? Where? How this is applicable to automation in selenium”. The chapter, as in others mentions “In this chapter we shall do …..” I would have expected the book to cover – why part of these activities. I would have liked to see “how these activities fit in overall landscape”. Usage of pattern matching in general has been explained well. Use of “exact”, “glob” and other nuances of regular expressions, is covered in detail.

I will cover in next posts remain chapters of the book … Do Check back

A Paradox of Reification – A Tool or a Fallacy?

Reification – Making some idea into a thing [Wikipedia]

Reification is a process (or is an essential element) of modeling – a process of reducing a problem involving multiple variables into single or few variables (For example say “drinking some brand of health drink makes you grow high or makes super intelligent”). If you see a complex (say meaning something that involves many interacting variables) problem made simple (reduced to a single variable problem), be sure to find “reification” hiding around. I would say it is a form of “reductionism”

Strangely enough, the life of Sales/marketing professionals, I think, involves nothing but reification. They can’t sell anything if they become aware of reification and start thinking about model vs. reality. A salesman selling insurance policy would say “Buy this policy – your health is secured – health is reified in terms of money you will get to pay for illness as and when it happens.

Politicians are another group of people who live by reification (I am not sure how many of them are aware of this term, though they might be aware of its effects). A politician is constantly required to create simple models of social life and impose them on his target electorate so that she can win votes. How else you think one can make promises to bring social equality, eradication of poverty by distributing rice at 2 Rupee per Kilogram or provide 70% or so reservations to certain class of people? It is reification @ work. Can any politician dare to think about (giving up) reification fallacy?

See the paradox in the act of reification. As a manager you need to support (and possibly create many new ones) reifications like “Continuous improvement”, “Customer focus (yes this is a reification), “SMART goals”, “Consistent Processes”, “Best practices”, “Metrics”, “objectivity”, “Predictability”, “knowledge transfer”, “business value”, “customer satisfaction”, “transformation” and so on. But when you become the target (rather a victim) of such reifications – you suddenly become aware of the problems of reification – while you may not know or identify by this name or label. How many of us and how often we disagree on appraisal ration ratings given by our managers (yes – your appraisal ratings are biggest and most influential reifications in our professional/corporate life).

Try asking a director, CTO, Head of Testing of a software organization – what is testing? He is most likely to talk about best practices, consistent process, business transformation, customer focus, continuous improvement etc. This person needs to reify ideas and abstractions so that he can do his job. He would not let complexities at the deep-down disturb him and simple picture is what he needs to work with.

Now, note that reification is not simplification. It is a process of abusing “simplification”, resistance to think beyond the world of abstractions. Thus, depending upon which side of the fence you find yourself in most of time, reification becomes a tool or a problem or fallacy.

In short the paradox is – you can use reification to work (mostly managers and people who need work with simple abstractions) as a tool. When it bites back to you as a victim (as in appraisals) – you would hate it and fight it as a type of “logical fallacy”.

Why wouldn’t Sales/Marketing folks think about reification as a fallacy? Do they?

In Pursuit of Quality, let’s Call Quality as something else

The theme of this year’s Euro STAR conference is “In Pursuit of Quality”. A great theme but an ambiguous I would say. I wonder what could have been the meaning/interpretation of the phrase “Quality” here. One might say “your own definition of quality” – in a group, it is a problematic one with the missing context.

It is interesting to note many in software profession acknowledge that the phrase “quality” has many meanings – different people at different instants of time paint/experience quality differently. Many admit that objectification of “quality” should be avoided. Yet, it is a rampant practice to portray “quality” as having some universal objective meaning that everyone agrees. Popular uses of the phrases like “Quality Software”, “Cost of Quality”, “Quality Engineering” etc., are examples of such practice. For many, “quality” is just a bucket where everything related to expectations of the users can be lumped together. It is one term that evokes a great array of human emotions from extreme pleasure to awe, from frustration to sense of achievement.

Let us imagine “quality” in terms of how Quantum Mechanics describes quantum states of subatomic particles. This “quality” is a like a subatomic particle in quantum superposition state and represents many meanings (or states) at any given context (or Einstein’s space-time). When one makes a swipe at this term and uses it with respect to a context saying “this is quality stuff” or “this quality is unaffordable” or “quality is free” – the wave function collapses and a particular meaning of quality shows up.

As Michael Bolton explains while posing a testing challenge about agile teams -

”For example, in a particular context, if we want “quality” to mean “bug prevention,” let’s say precisely that. Then let’s recognize the ways in which certain approaches towards preventing bugs might represent a threat to someone’s current interests or to their personal safety rules. If, in a particular context, we want quality to mean “problems solved for customers”, let’s say precisely that. Then let’s recognize that there are many approaches to solving problems, and that some problems might be solved by writing less software, not more. If we want “quality” to mean “many features in a product”, let’s say precisely that. Then let’s recognize how “many features” can satisfy some people—while adding complexity, development time, and expense to a product, thereby confusing and annoying other people. In other words, let’s use the word “quality” in a more careful way, which starts with deciding whether we want to use the word at all.”

That is to say if you want to say something to indicate quality – say it (without using the term quality). In this way, you can prevent misuse of the term for the better. If you want to pursue quality - say what you exactly mean so that people can understand you without confusion … Probably people will stop using phrases like “Cost of quality” or “Quality plan” or “Quality Assurance” or stop using quality with another set of abstractions like “Creating business value through quality software”

To repeat Michael “Quality isn’t a thing, but rather a complex set of relationships between products, people, and systems. We should be calling quality as something else that states what it is in a context rather than calling it with one word always.

I have a suggestion. Instead of using QUALITY – let us use words made of up characters in the word “QUALITY. For example - you might say LITYQUA of this software is poor to indicate usability or you might say TUAQLI of this software needs to be worked on to indicate security or cost. Once thing is sure to happen when you do this – people will say – what? When you explain them the meaning or interpretation of attribute that you are trying to represent is using the “jumbled” up word – they might relate it to that one thing and only that thing. Thus misusing and lumping “all” in to Quality, stops – I hope.
Try saying some other term when you want to say “quality” – difficult?

Two versions of being practical...

My boss tells me always that my ideas about testing are impractical and esoteric. I find it hard to understand why he thinks so. I believe my testing and automation ideas are as practical as one can get because I can demonstrate the practicality of the ideas by "doing" testing or automation. I suppose being practical - should be mean "demonstrable". I can show how my ideas of testing can be and are practical. Boss is not impressed.

Probably, my boss has another version of practicality. It goes something like this. Regardless of what subject matter or technicality of testing as it is practiced in real time - there is a social, political, business side to it. Many things that testing practitioner believes to be true are not possible in real world due to political and non technical aspects around testing. Under such situations - being practical means changing your testing philosophy and approach to suite the context of the testing world. Harsh word for such adaptability could be "compromise". If you are surprised or questioned instances like stakeholder seeking quick benefits of automation (something that reduces cost of testing), counting bugs, counting requirements, using bug metrics to measure testers, expecting testers to find all bugs, doing 100% testing - you probably are practical in another sense.

Living with many irrational notions about testing and trying to do what is possible, taking the world as it comes without challenging it or fighting the problems is the version of practicality - probably my boss believes.

Both of us could be right or wrong. How do I know?

This reminds of me of a Jerry Weinberg's saying (para phrase) - "Instead of calling something as irrational or illogical - consider it as rational or logical from another perspective or set of values".

You would like to call "testing" as an engineering pursuit? Wrong... it is becoming (it was from the beginning but now it is showing up more clearly) more social. Testing works in social systems and is affected by cultural, emotional, economical issues.

Time to start studying economics, social science to reason behavior of people in testing context. It is not rational at all times....

Shrini

What type of tester are you?

I was assisting homework of my 7 yr old daughter in her science assignment. The assignment was to collect pictures of living and non-living things and make a collage. I thought about how one could approach this work and have fun while doing it. Being a tester, it is hard not to see connection in a work like this. Here is what I thought….

Let us say two testers “A” and “B” are given this homework assignment of creating collage of living and non living things.

Tester A: Living and non living things? Just collect few pictures of animals, plants, humans - paste them as living things and collect pictures of cars, buildings, mountains, bridges and paste them as non-living things. Home work done. Test pass and on to next thing…

Tester B: While thinking about traditional ways of looking at living and non living things, this tester also questions his own understanding and meaning of “living” and “non living”. He looks around, reads, discusses with colleagues about these things. He, then ends up with a broader list and accompanying narration.

What is the difference between these ways of approaching a task at hand? Tester A is more focused on “completing” task with minimal mental engagement about the settings of the task. He takes “widely accepted” meanings and understanding of the things and optimizes the details of the task. Tester B, starts with questioning the premise of the task, meaning and understanding of the things required to complete the task. He seeks a broader understanding of task elements and attempts to question biases, prejudices of his own and others around. Though on the face of it, it might appear that Tester B will take longer time to complete the task (homework), through practice, he might able to finish the task nearly at the same time while understanding of broader aspects of the task and clarifying biases and assumptions.

Today’s testing industry promotes type A testing mentality. Type A testers eventually will become “commodity” (with price tag 20 Dollars per hour – can execute 30 test cases per hour etc). I am not sure that is a good thing or not. If you are a manager or someone who manages testing as business – you would probably have (or would like to have) more of Type A testers as such people are plenty. Simply, take truck load of freshers from college, run them through 15 days of training on testing vocabulary, make them memorize some domain stuff (banking, insurance, telecom etc) , teach QTP – there you go an army of (Type A) testers ready for deployment (hence billing).

If you are a tester looking at long term fulfilling career – you would want to take path B – practice to do things differently, challenging status-quo, thinking broadly about the problem and its context etc. To become “Type B” tester you need practice – practice of doing testing (weekend testing is an excellent platform or training ground). I am afraid there is no shortcut here – no 15 day testing courses to get there.

There is a catch here… Type B testers are mostly rebels and free thinkers and often want break status quo, create new paths, explore new territories. Much of our industry is not very nice with such folks. Very few can tolerate “ever questioning”, skeptical and rebellious tester in the team. In a control freak, military kind of set up where key focus of the job is to “follow instructions” or “getting job done” – Type B testers get frustrated. Some get converted into Type A or leave the organization. Sad situation but true in many cases. I hope that one day either rebel testers will learn to live harmoniously in a hostile, confirmatory environment or industry recognizes the value of these “problematic” folks and make space for them.

“Testers are like lighthouses at sea shores, headlights of an automobile” says James Bach (paraphrase). Good testers, apart from completing the task given to them, engage in parallel about “meta” things for better and deeper learning. That is a good thing (can I say for testers not for folks who manage testing as a business?).

Did someone say “Type C” tester?

What does that mean?

The other day I visited a science exhibition at my daughter’s school. As I was passing through an array of models and demonstrations, I could see every student was enthusiastically trying to draw every visitor's attention to his/her piece of work. Everything sounded good except one thing - the students were seen literally reading some sort of script to explain their work. Some smart guys memorized the stuff well so they did not require any hand written (mostly) note to assist them delivering the script but many did keep one by their side. It was hard to find someone who “knows” what he/she is trying to explain rather than forcing the stuff through using pale dialogue delivery.

There at the corner, a bright spectacles boy waved his hand calling me to see his work. I walked up to him. He was demonstrating “law of conservation of momentum” through a model that roughly looked like the one above.

After a customary greeting, he read through his memorized statements about law and looked at me with hopeful eyes asking if I understood anything out of it.

I asked him “what does this law tell you”.

He promptly, this time with more energy and desperately attempting to sound confident, repeated the law (the statements) like a tape recorder.

What do you understand from the term “momentum”, I asked him.

He was totally unprepared for the question (missing in his notes - missing test case?)

He quickly recovered gently tapping his spectacles said “It is simple sir - it is mass times the velocity”

I know that is a mathematical formula for momentum.

Student finally found something that I agree with him and said “Yes. you are right sir”.

At that instant, I remember my hero Richard P Feynman - the (most) curious character and asked him “But what does that mean? Mass times velocity?”

This time, student was not expecting that question at all.

I moved to another demonstration where they were showing double helix model of DNA and the girl explaining the model, gave a 5 minute non stop explanation of what DNA is and what genes are.

Again, some question “What does that mean” to the explanation “genes carry genetic code so that characteristics are passed on from species to species” got everything to a grinding halt.

I became pretty notorious from that day in among my daughter's friends about asking such silly questions about things that are taken for granted for the students. My daughter later told me that her friends were impressed with my “latest” knowledge of science and my questions.

Later, over the week end, one the students (harassed by me in the exhibition) paid a visit to my home apparently (I thought) to take revenge on me. He, incidentally was my daughters classmate and asked me “what do I do for living”. I said to him “I am a software tester”. While he did not understand that very much, he wanted to know how learn things. I explained him about few things that I learned of late as a student of software testing (than when I really a student to learn science). He appeared to have understood some of my approaches.

When he was about to leave, I gave this book “ Surely You're joking Mr Feynman” and told him - "learn science like this guy". I was happy that I introduced Feynman to a student hoping the “Feynmanism” spreads in the school.

I wish a very happy and prosperous New year to you all

Shrini

Fish baking story ...

This was a story that came to my mail box.

A little girl was watching her mother prepare a fish for dinner. Her mother cut the head and tail off the fish and then placed it into a baking pan. The little girl asked her mother why she cut the Shead and tail off the fish. Her mother thought for a while and then said, "I've always done it that way - that's how grandma did it." The girl was not satisfied with the answer, and went to visit her grandma to find out why she cut the head and tail off the fish before baking it. Grandma thought for a while and replied, "I don't know. My mother always did it that way." So the little girl and the grandma went to visit great grandma to find ask if she knew the answer. Great grandma thought for a while and said, “Because my Grandma told once that the baking pan was too small to fit in the whole fish”.

In most cases we fail to challenge belief systems and assumptions.

This is true about many ritual centric processes that we are asked to do in testing today. If you dig deeper, you would be surprised like this little girl. There are no real reasons to do certain things. Typical answers might be “my boss wants it that way”, “this is how our clients want” or “this is how I have done it always” or “this is in that book” or “this is how the consultant who we hired told us” or “this is how it always appear to work”.

One trademark quality of a skilled tester is skepticism and ability to think beyond rituals – ask “why I am doing what I am doing”.

Are you asking this question frequently in your job as tester?

Shrini

How can Mathematics be the Language of Nature?

I am reading this wonderful book about Physics (and history and philosophy of Physics) "Tao of Physics" by my all-time favorite Physicist and author Dr. Fritjof Capra, a contemporary of Werner Heisenberg. Incidentally, Dr. Capra's other famous book "Turning point" has really introduced me to the concept "systems thinking" while I struggled to grasp the subject by reading another influential legend and Guru of software (testing) Jerry Weinberg. James Bach and Michael Bolton introduced me to the subject of "systems thinking" as science of complexity.

Well... coming to the point, this post is not about system thinking but about a paragraph from the book "Tao of Physics". Here goes the text ...

"... The words of our language are thus not clearly defined... Science, on the other hand aims for clear definitions and unambiguous connections. Therefore, it abstracts the language further by limiting the meaning of its words and by standardizing its structure in accordance with the rules of logic. The ultimate abstraction takes place in mathematics where words are replaced by symbols... In this way scientists can condense information into one equation, into one single line of symbols. The view that mathematics is nothing but extremely abstracted and compressed language does not go unchallenged...mathematics not just a language to describe nature but is inherent in nature itself. Originator of this belief was Pythagoras, who made famous statement - "All things are numbers" …

This puzzles me ... considering(or conceding) that mathematics is an abstract language used by scientists to represent theories, models and scientific thought in a precise and unambiguous way, how can such an abstract language be suitable to describe nature (even in parts) ?

Nature is live, rich and multi dimensional where as mathematics is at other end of the spectrum - abstract, precise and single dimensional. Did Pythagoras and for that matter Galileo make mistake by saying "book of nature is written in the language of mathematics". I am certain that mathematics does not be to be interpreted where as nature and its manifestations need to be. Am I making sense?

Nature of Problem(s), we testers (should) investigate (or solve)

I am puzzled about the nature of problems that software testers, today are juggling with or should be losing their sleep over - more than the problems themselves. As humans, we love simplicity (not sure about about other species - not sure if there is any way to check if they love simplicity too). In every walk of life, we seek and embrace simplicity. Many experts tell the stories about how they have solved mysteries of the world through and give us the tools, techniques to conquer/dominate the world around us. We now have quantum computers even though there is barely anyone who has fully understood Quantum theory. We can make machines, apply our partial knowledge but make a mistake of claiming glory of scientific knowledge saying “this theory is true as I have experimentally verified it and there are practical applications”. The mistake that we make is about “completeness of knowledge” – making a machine based on a theory does not amount to complete understanding of the theory.


Alfred North Whitehead, the famous English mathematician and Philosopher warned us “Seek simplicity and learn to distrust it”. Simplicity is so tempting that it fools our minds and thinking and makes us glued to it. It appears to me that we have not learnt to distrust simplicity at all – worst many even does not know that we should “doubt” it. There is “banana” principle, attributed to Jerry Weinberg. For many things like simplicity, we have only learnt how to start but have horrible lost the trick “when to stop”.

Many in software world believe that software testers are expected to answer questions that, simply lead to “pass” or “fail”, “works” or “does not work”. These questions are like asking “is 4+4 =8? Is this an isosceles triangle? Is the temperature of this body 86 degree centigrade? What is National Stock market Index today?” Engineering, mathematical and scientific questions right?


In our software profession, we love to pose “engineering/scientific” questions that lead to binary answers. After all we are software ENGINEERS and software is ENGINEERED. Hence I started this post with simplicity and our obsession with simple models of complex things. What is the true nature of questions/problems that software testers investigate and solve? Let me make a bold conjecture here. I think they are similar to the questions that we pose in the realm of social sciences. For example, questions like “Is homosexuality linked to or mainly due to some mental illness?” or “Should burqa be banned in public places” or “Is there a God”. These questions are asked in a broad social setup and are linked various material practices of the people in the society, about their culture, history etc. Much in a similar way, software operates in a social context and people are mainly concerned about how software alters their day-today work. The nature of questions/problems requires non-binary, qualitative answers and solutions. The questions about quality are based on emotional, subjective feelings (there is no binary 0 or 1 about software quality).

As Michael Bolton says “decisions about quality are always emotional and political”. How can there be binary mathematics in emotions and politics? That is the point these software engineers and process people miss and insist on quantified definitions of quality and other abstract entities associated with Software. It appears to me that they are constantly conditioned by the business environment to believe so. Coming back to simplicity, software engineers (process/CMMI/Six Sigma) people definitely think that the problems that software testers/engineers solve (investigate- actually) are purely engineering/mathematical in nature. Hence they believe in absolute answers, causality, objectivity and certainty.


I think they are wrong !!!! When do we learn when to stop saying “banananananana..?” While you are reading this and interested in debating about “nature of testing problems”, many budding software testers are happily busy in churning out hundreds and thousands of test cases and blissfully shouting “pass” or “Fail”. Well, you might say do they have options … I am sure they do.

Shrini

A movement called weekend testing – What it can do to you and your testing career?

An extremely contiguous phenomenon is sweeping in some circles, communities of software testing; it is called Weekend testing (twitter @weekendtesting). Presumably, born out of few young minds (Ajay Balamurugadas, Parimala Shankariah, Sharath Byregowda, Manoj Nair ) and influenced by context driven software testing philosophy. In a short span of time, it has traveled from Bangalore (the home of Weekend testing) to Europe and the US. So it is globalized movement. As testimony of its popularity and the value, who's-who in global testing community have acknowledged it

It is hard for anyone who is a passionate/skilled software tester not to be associated with it. Once you are in, you keep going and loop in others. Such is the power of the concept and the way weekend testing works. I would probably equate (not mathematical sense) weekend testing to open source software development. What drives these movements? Passion, Desire to learn, Demonstrate skills in the real time "On Demand". The last point is something that is more apparent in weekend testing movement.

What makes weekend testing - a movement or a revolution in its infancy?

The objective:
Practice testing (or software testing) in real time with like minded (I wish if some dis-like minded folks join it – we need fight confirmation bias always as testers. Being vigilant about fallacies of "you will see what you want to see or you will fall in love with that which confirms what you know").

This objective is very powerful and has enormous potential to grow big. Keywords here are "practice" and "real time". Now you can see testers saying "I was practicing testing" just like a musician doing is "Riaz" or the sportsman sweating it out on the ground. A manager would call few of his low performing testers and say "Go and get some practice of testing in week end testing sessions". While his testers are training, the manager can check their progress by viewing the session transcripts and related blogs posted elsewhere. Transparency to the core!!!

The structure

Here is a generic format of how a weekend testing session is arranged – more detailed description can be found here

A group of testers sign-on to Skype or Google talk and get started with their testing at a specified time as announced on the weekend testing website. One of the testers assumes the role of facilitator giving the mission for the session and moderates the whole thing. Mission of the session is itself tested first – meaning it is scrutinized for possible ambiguities and a refined one is adopted. Testing starts bugs, issues are logged in an open source bug database and followed by a debrief session. The session transcript is posted with results for everyone to see and learn further. It is almost free for those that have computers and internet connection. Weekend testing thrives on usage of open source/free tools. It is more or less self-driven – all that was required to set up an initial structure for testers to get started and posting the results and transcripts of the sessions.

It is a terrific place to learn With so many willing, skilled and available testers and coaches around what you are learn is nearly unlimited. With a wide variety of products to test, variety of testing missions to try out – there is something new to learn in every session. With people across the globe participating, you get access to an interesting cross section of skills, cultural legacies to learn from. You can improve your communication, writing skills in a friendly yet mildly critical environment. People here tend to me cooperative and more interested in seeing everyone succeed. It is the genuine interest in other's growth and improvement that makes weekend testing a unique platform for greenhorns to gaze around.

It promotes "community" certification/acknowledge - not given by any "standards" like bodies. Needless to say the people associated with the movement are known for their "critical thinking" and would go to any extent to be "open for scrutiny/challenge any time anywhere". That makes this movement strong and really unbeatable.

It makes testing demonstrable in its deepest details possible. For long, people influenced by context driven testing have struggled to explain the power of exploratory testing. They have been trying to argue for skill in testing as opposed to so called best practices and process driven testing. And there "test cases" the ubiquitous building blocks of testing and the questions that were often asked "how can you do testing without test cases, How can you repeat testing? How will client accept that we tested?" Now we have a proof in a public movement and "open for scrutiny anyone anytime" – You can literally see testing happening in front of your eyes. What more evidence you need?

The People

The founding members of this movement are, at the core, very dedicated, passionate software testers whose sole distinguishing qualification is their appetite for learning and become skilled software testers. That drives them to do things that they do. They are constantly building their reputation through demonstrating their skill and learning constantly. So, hanging around with these people can be very dangerous. They will make you like them. The weekend community is very aptly mentored, coached and supported by people like Michael Bolton (no… not the singer or the "officespace" character), James Bach, Cem Kaner, Pradeep Soundararajan, Matt Heusser, Jerry Weinberg, just to name a few.

We (notice, I am using this "we" after a long hesitation as I have not contributed enough to include myself in this movement) are thankful to these leading lights for encouraging the movement and advertizing it.

The Future of weekend testing – The way I see it.

Few thoughts …

• There will be many chapters of weekend testing all over the world. There will be weekend testing conferences to the scale of STAR conferences in the US.
  
• Big universities teaching software engineering will recognize this movement and will have representatives from them
  
• Commercial product companies will engage weekend testing folks to test their products. With the support of crowd sourcing companies like utest, week end testing will gain authenticity and grow.
  
• Weekend testing will then get into weekdays as people want more of it – we then have to look for another name to call and identify ourselves … But that is an interesting problem to solve
  

I am looking forward to participate and contribute this movement …

Update (10th March 2010)

Parimala Shankariah has brought my notice that Santhosh Tuppad, yet another promising testing star in the indian software testing horizon, who was recently featured on utest - has been a very contributing member of this movement. Thanks Parimala for point this out ....

Santosh ... nice to know about your contributions.. Keep going... Look forward to participate in sessions with you.

Shrini

Man and the machine ....

Can a software be produced and maintained using a machines in a factory … like Google does?

Notice, with their machinery and factory setup, they have been able to cut down the testing cycle to 4 minutes... Sounds unbelievable .. I think it is ..

Google produces Innovation machines and IT vendors set up factories… So back to Factories and start oiling machines. Probably we should change IT designations to map roles on assembly line shop floor … - foreman, supervisor (this is there already in some sense), operator, inventory manager, assembler…. As few expect, IT work will eventually be a factory job. For one, my wife and kids would be happy. I can then work in shifts and come back home at sharp, say 5:30.

It is interesting to watch what Google is doing ….though their innovation factory recently had an outage…. With “Buzz” having “bugs” related to privacy...

You know what … as a tester, bugs make me happy… that is an eccentric , one sided view …especially with those who boast “machines and factories” of software. After all we are very far from the day when truly software will be produced in machines, requirements will be read from humans through bar-code type readers, we would be able recall software as cars (instead of sending a patch) and there would software garages.

Dreaming …!!! Someone woke me up... It is a brand new Monday….

Shrini

I’m blocked… but why?

"There is no such thing as writer's block for writers whose standards are low enough." American Poet William Stafford

I am blocked from doing any interesting writing for over 2 month or so. And this post is all about the condition of being blocked in writing.

As paradoxical as it sounds, I have not been able to write anything to my satisfaction so that I can publish. In the world of writing, they say it is a condition called "writer's block". It's a terrible feeling for me as block keeps haunting on every attempt to complete a half finished work or a new one. Here I am sitting with over a dozen of unfinished posts, like a kid confused about which toy to pick and play. It is funny that not long ago, I was chatting with Parimala Shankaraiah, a (or the?) curious tester about this condition called "writer's block". I suggested to her that when blocked, try writing about it. That is what I am exactly doing now. To illustrate the influence of WB on me, I am planning to keep this post small and focused on the topic and close it as soon as possible.

For non starters, WB is a temporary mental condition of extreme self censorship leading to rejection of semi finished parts of writing as this reference suggests. Keyword here is self imposed censorship. Those getting stuck in WB, can take conform in William Stafford's statement consoling themselves with pat on their backs for their high standards of writing.

Why I am blocked on writing?

While I am not short of ideas to write, when I pick up something to write after writing few paragraphs, I start getting a strange feeling that I am repeating or the plot of writing is hypnotically drawn into a predetermined destiny. That is where I stop writing. Not sure what is happening.

WB, like inattentional blindness appears to occur when you are ultra/super focused on a topic so that you literally stop thinking. I am not sure, my state of WB has anything to do my focus (or lack of it) on writing. As defocusing tactic, I switched to twitter and tried my hand there sufficiently enough to get an attention from none other than Jon Bach. That still leaves me high and dry and I am back to WB. Another thing that I suggested to self – stop writing but speak (do pod casts) or draw/paint (cartoons etc). I need to try these channels to see if they help. I think what happens in WB is one (of several) channels of expression of thought gets blocked while keeping others intact.

One thing is sure; I am under pressure of not being able to write even though I have many "good" ideas to write. Some say WB leads to lack of motivation or happens due to lack of motivation. I am pretty sure that is not the case with me. Anxiety, fear of being ignored or failure certainly is back of my mind.

Aiming for High standards? High degree of censorship? Yes, that appears to be the case. I reject lots of writing (pile of trashed pieces of writing) as they don't qualify my own criteria to be good enough to be posted.

I don't agree to generic prescriptions like Lack of preparation, reading, and picking up boring topic leads to WB. My experience of WB is that I am blocked on those topics I deeply interested in and dying to writing something significant, lack of preparation/reading might be an issue in some cases but the in case of the ideas that are pretty original, I often avoid reading to get my flow of my "unbiased" ideas.

Well, now I am suddenly realized that I am supposed to suffering from WB…let me abruptly end this post and start mulling over the cures for WB.

Oh!!! What a relief. This post comes to an end. Have I overcome writers block? Over to you…dear reader!!!

Note: I am surprised to note that there are other types of writer's blocks such as this and this. Here the word block is intended as "collection" or "store" of useful writer tools. Same name different connotation.

Why GUI Test Automation is Popular and Tempting? Part I - Business context

It is rare to someone on testing in IT world not be aware of "QTP, WinRunner, Rational Robot etc ". For the most in IT world, the word automation is synonymous with GUI automation. Perhaps some even might ask - "Are there any other forms of automation other GUI automation". Lots of how IT world sees automation has to do with how they see testing itself. You might have heard this "celebrated" statement - "Regression testing is more suitable to do automation (read GUI) so that we don't have to have our IT staff do that unnecessary testing manually".

To my surprise, having been associated with IT for many years now, people here still believe in perfect requirement, perfect test case and hence "doing it right first time". In contrast to this, software product world (likes of Microsoft, Google) treats test automation and hence testing in a total different way. These two probably operate from two extreme ends of automation spectrum. While discussion how automation is perceived and realized in IT and software product world can be a separate post in itself, I propose to elaborate here my views on the popularity on GUI Automation in IT.

Testing is necessary evil and should get done quickly

This is the reason #1 that drives GUI automation in IT. Business spends on getting IT application features developed and hence often fails to see a reason why they should spend on "testing" at all. Business feels that IT should get their acts together, get aligned to business (hence IT-business alignment is often a well understood but poorly executed theme) and use the money towards delivering business value. I even have heard some business folks saying "We pay for the application features not for their testing". Hence funding testing becomes IT's problem. That is why they look for solutions to reduce testing (or spend on it). As simple as it sounds, IT pays for testing (not the business)

Testers (read business users or analysts) are hard to find and they hate regression testing

Traditionally, in IT business analysts (aka BA's) are hot commodities. They do most of what we call as "testing". With outsourcing wave sweeping IT, some of BA roles are taken by either BA's from outsourcing vendors or some junior tester working under supervision of BA through highly scripted test procedures. The problem is that these BA resources are few and are in great demand. But as software releases happen, changes come in – testing (regression testing) has to happen and BA's hate regression testing (or all of testing). So, an IT manager needs to find a way out to carry out this regression testing somehow.

How to get testing done fast and cheaper?

If you want to do something fast, give it to machine or a computer program - goes a typical approach of an IT manager. So automation could be the answer. If you position automation as an aid to reduce the cycle time to market chances of you getting funding is very high. Surprisingly this claim is so powerful that when made often goes unchallenged. The magic word "reduction of cycle time" (that too without any qualification) is tempting for many that it will make them to completely overlook bottlenecks in development (programming), bug fixing, investigation and any other non-testing activities. Thus speaking in terms of business language is so important in proliferation and apparent success of GUI automation in IT.

With GUI automation, you can easily make a business case for automation.

To me, the terms like test cases, regression testing, test cycle time etc are the ones that business stakeholders appear to understand very well. As compared to unit tests and other forms of non GUI automation, you can make your case for investment in GUI automation to business stakeholders. Let us say you have 5000 test cases for a regression testing to cover that takes 5 person days to complete, automate 50% of them you can straight way knock off 50% of your testing cycle time reduced. This claim due to its simple math and logic makes a compelling case for GUI automation or simply "automation". What else can be more appealing than investing once in automation (for the sake of making the business case and further forever – down play the maintenance costs) and saving on a percentage proportional amount of automation achieved for 100's of cycles in the future?

Tool vendors help you by talking in business language

Enter tool vendors – with the fancy looking charts, scary quotes like "more than 60% of project cost goes to testing, what are you doing to reduce it" and likes Gartner's and Forrester's to support the claims with numbers, life for an IT manager in charge of testing has never been so easy. Just buy a tool, engage an outsourcing vendor to do GUI automation – all your testing worries are over in one go. Dreaming never ends here. Take a look at any brochures, websites, commercial literatures – they are full of business terms like ROI, Cost of testing, business impact, and Time to market and so on - making a perfect connect with those that matter in making decisions about spending.

Now try doing this (or these) with non GUI tests such as API tests or xUnit framework based unit tests – what will business say? What will IT managers say – "well, developers got to do that testing anyway"? So there is no apparent business case to make so sell automation. Writing unit tests will not speedup your testing cycle, will not reduce time to market, might reduce some portion of that "boring" regression testing – but. So you have no financially oriented incentive to do any automation that is non GUI. Since GUI automation paradigm as so much hardened into dogma – very few challenge it.

If you are the one working in a software product company – you would say "do less of GUI automation, they are very brittle and costly to maintain". You would be surprise to know/hear – the opposite works (or at least appear to work) in IT world.

It is "power of talking in business language, very few challengers for the claims made about automation and excellent support by tool vendors" that makes GUI automation popular and tempting to attempt. Even if you fail – you can blame it on almost anyone or anything around – test cases, tool, automation folks, application changes, changing requirements, test data, operating system patches and so on.

To be continued in Part II