Cause and Effect - Non Linear Systems

Here are three examples where cause and effect do not appear to corroborate. Take a look.

1. Build a flyover on a busy road hoping that traffic will ease – A personal experience [Traffic will actually increase with flyover] 
2. Dip a thermometer in boiling water. What happens to temperature reading? – Adopted from Gerald Weinberg’s Book “Introduction to General Systems Thinking” [Thermometer will show a lower temperature reading due to difference in thermal expansion between mercury and enclosing glass tube] 
3. Making Cars more safer will cause drivers to become more aggressive and rash – “Peltzman Effect” Adopted from freakonomics post "What happens to your head” 

 In each of these cases – the effect or what is expected is not what happens – but the opposite. There can be explanations – lesson for testers : think holistically, develop systems thinking mind.

Why this happens? I think we often approach in terms of or use analytical/reductionist thinking - just bread/divide a thing into its constituents (atoms) and study them. This linear thinking of single cause-effect (taken one at a time usually) can help understand some aspects an object/phenomenon. With non linear systems such as societies, political/cultural systems, business systems etc - this simple cause-effect thinking simple does not hold good. So - think in terms of systems and their interactions.

Shrini

Do statistics lie?

There are no whole truths; all truths are half-truths. It is trying to treat them as whole truths that play the devil" – Alfred North Whitehead

I received an email this morning with this topic of statistics and the content of mail and overall theme of the mail boldly proclaimed “statistics do not lie”. When we speak of statistics in software world, we typically refer to metrics and various numbers representing effort, number of defects, test cases, cost etc. So, instead of talking about statistics – lets talk about “Do numbers lie?”

We have two items here – numbers and lie or truth.

Let us start with numbers. What is a number after all? The need of having numbers probably is related (or caused by) need for counting. In Egypt, from about 3000 BC, records survive in which 1 is represented by a vertical line and 10 is shown as ^.
According this historical account, Pegan priests need to calculate the frequency of natural phenomenon. One of the best known examples of this period is the Stonehenge stone circle in Britain, built by the Druids as a kind of celestial observatory in 1,800 BC. For cave men of pre-historic age, counting facilitated sharing food items - probably. The pictures on the caves and archeological finds give us an idea that people in those days counted by drawing lines to indicate “how many”. Counting fewer items, let us say 6 fruits could be fitted in this idea of counting with fingers or counting by drawing lines. But how would you count number of fruits in a tree or number of people in a village? Discovery of place value system allowed counting items in 100’s and 1000’s. Hind-Arabic numbers 0…9 were known since probably 300 BC. Before Hindu-Arabic numbers, people used “roman symbols”. Interestingly enough, Greeks and Romans did not know the idea of “place value”. And human civilization evolved.

Gonitsora – an initiative from few students of Tezpur University India, carried an article on history of counting that said –

“The first motivation for people to create number was the human desire to the manyness of a set of objects. In other words, to know how many duck’s eggs are to be divided amongst family members or even how many days until the tribe reaches the next watering hole, how many days wills it be until the days grow longer and the nights shorter, how many arrow heads do one trade for canoe? Knowing how to determine the manyness of a collection of objects must surely have been a great aid in all areas of human endeavor.”

When we say “9” what does that indicate in a purest and objective sense? Nothing. OK, let us say 9 cars? What does that mean? Extending it, “9 cars parked in front of a house” what does that mean? “9 cars parked in front of house of a celebrity in London”. You might say “might not be any significant or interesting” as it is common for a celebrity in London have that many cars. Now if I say “9 cars parked in front of a house of politician in Delhi” – Something surprising or some planning happening on a political discourse. Now if I say “9 cars parked in front of a poor in a remote village in Somalia” – what does that mean? You would really get interested to know what might be happening in that house, who came in those cars, where did the come from? Did this poor steal those cars and all sorts of questions.

Pause for a while – and think did or do the number 9 revealed any truth in each of these situations? Is the number 9 capable of telling any truth or for that matter lie at all? A number meaning full and relevant by set of object/objects, people, ideas, events that the number points to. Thus it might be totally meaningless and absurd to say “number don’t lie”, as a matter of fact numbers of incapable of telling truth or reality independent of context, observers and recipient of the information.

Let us talk about truth. What is truth – a question that is at the base of all philosophy, science and every root of what we call “knowledge”. For the purpose of this post, let me use this definition (very tentative, provisional) - “truth is a qualifier that we can attach to a piece of information about which a group people do not disagree by and large”. Back to software world – give me an example of truth. One might say “In this month there were 9 sev 1 incidents in production”. Is this a truth? You may point to live incident tracking system and show a list of incidents reported in this week and say “look here is the truth there are 9 live incidents”. Let me apply my “provisional” definition of truth here. Let us call 10 people – few programmers, few business analysts, a project manager, a business unit head, a customer and a sales manager. Let us put these people in 10 separate rooms and show them the list of 9 live incidents and ask them “is this information true?” Let us record each response. What do you think would be those responses? Will all of these agree on the notion of truth of 9 live incidents? I guess – many would say “Yes, I know there are 9 live incidents this week But ……” What follows after but is each person’s view point or story of how they view (defend, attack, frown, shout, feel sad etc) those incidents. How do you extract “truth” from this beautiful, “god-like”, impartial number “9” quantifying live incidents”?

You would soon have a consultant selling a version of “cost of quality” and attach some dollar figures to these 9 incidents and sell a multi year “transformational” deal to reduce the cost associated with these incidents. Should you believe him?

Often when executives say “I need statistics, numbers” – it seems to me that they are really (should be) interested in the stories behind those numbers, they are (should be) least bothered about numbers themselves. Numbers are masks for stories, events and emotions that they represent.

Numbers, statistics – are incapable of telling anything in absence of context, stories, people and their motivations. For now, I can say the issue of whether statistics tell lie (or truth) is settled – they don’t tell anything.

An exercise: When I was preparing this post, a colleague of mine, Joy Chakraborty challenged me and said “company financial results” are objective truths about company’s performance (he did acknowledge Satyam Saga and other irregularities about how company financial results could be manipulated). He simply asked how the numbers in the statement - “Goldman Sach’s reported net earnings for Q2 2011 – of 1.1 billion USD - 77% up over previous year’s same quarter” are not objective truths? What do you say?

Is Pythagoras theorem true? How about Einstein’s General theory of relativity?

When Testing Fails ...

Carl Segan once famously said “Science is a self correcting process – an aperture to view what is right”. Carl Zimmer in an article on Indian express says “… science fixes its mistakes more slowly, more fitfully and with more difficulty than Sagan’s words would suggest. Science runs forward better than it does backward.” According to Zimmer, checking the data, context and results of a published scientific work is not of much interest to journals that take pride in “first” publishing ground breaking new research. For scientists scrambling for grants and tenures – checking published work is not attractive and often an exercise not worth its effort. As a result of this, Zimmer says, original work/papers often stand with little or no investigation or scrutiny by peers. This surely, is bad for science. Zimmer, suggests the community to have focus on “replication” and setting aside time, money and journal space to save the image of science as self correcting pursuit of knowledge in Carl Segan’s words.

Well … does this have anything to do with software testing? I recall James Bach once saying “it is easy for bad testing to go unnoticed”. Like science and scientific community’s social fabric – software testing world (software world in general) has built-up layers of wrapping and packaging. It is easy to find some reasons in one or more of these layers in the ensemble of social systems leading to few missed bugs that cost few days of outage of say- a popular e-commerce website. Any query or investigation on the effect of outage or episode of missed bugs would set-up a nice blame game looping all the way to testers, developers, business analysts, test data, environment, lack of technical/business knowledge and host of other things. Like it happens in published science works – hunting down the culprit or group of culprits would be time consuming job. In any case it is regressive job and takes valuable resources from many “progressive” tasks. Right? In Zimmer’s terms – spending time on production bugs is similar to running backwards. Does testing work well when running backwards? Do stakeholders like it when testing is running backwards? Not sure if published research wrong or publishing a new perspective on the basis of an existing work can be as productive as hunting down a missed bug and trying locate the reasons for its birth in the first place.

While process enthusiasts, Quality assurance people, Sick-Sigma (oops… six sigma) people might protest and insist on full blown root cause analysis of all bugs reported in production. SEI/CMM folks might make it mandatory to document all lessons learnt from missed bugs and refuse to sign off the project closure unless this is done. In spite of this – in a fast paced life of software where cycles between conception and real-working-software, are shrinking, ever increasing number of platforms (don’t forget mobile platform) – who has time to look at root cause analysis reports and all those missed bugs?

I remember a business sponsor once saying – “I can’t wait for testing to complete, once the development and put the stuff into production. If something breaks, we have enough budget provisioned to take care of any production failures.” Here, failure to put (buggy) software in production is more disastrous than waiting for “perfect” software to develop. Back to layers of social system in software testing – it appears to easy to hide bad testing – unless you screw-up very badly, the chances are that your stakeholders will never notice what good testing can get them.

I often wondered looking at some testing that happens in some organizations – how they are managing to stay afloat with such bad testing? The reason is probably – when testing fails, it is difficult to attribute it to testing and call it as such. It requires courage. How many testing leaders are willing to admit their testing sucks without hiding behind fancy looking metrics, root cause analyze reports and charts? That is the effect of “software testing” as “social process”

Sure Ways to Reduce Test Cycle Time through Automation

World is simple, we complicate the world for the sake of it – says a friend. There is a simple concept called "automation" and another one "testing cycle time". Why these terms can not be simply understood without much fuss and inquiry? he often argued with me. This friend is a manager and works for IT services company. A constant pouncing on me by this topic of test-cycle-time-reduction-through-automation and related hype, frustration and feeling of achievement, made me to think deep about basic laws or golden principles that govern this phenomenon of cycle-time-reduction.For the benefit of my blog readers, here, I make them public. Read them, understand them, implement them and be blessed. A caution here any criticism and cynicism about these laws will have harmful consequences to the beholder. These are golden principles and axiomatic about test automation!!!!

First principle - “About Testing”: Strongly believe that software testing is a deterministic, highly repeatable and structured process – somewhat akin to a step-by-step procedure to produce say a burger or car. You have to believe that given a fixed scope of testing, it always takes a finite and fixed time (effort) to complete testing. Needless to say, you have to have absolute faith in testing processes and standards. Your faith in the power of processes and standards making testing predictable and repeatable is an important success factor. It is also necessary to abandon any misconceptions you might have about relationship (or dependency) between testing and automation. You should treat testing truly as a mechanical, step-by-step and repeatable process to ensure Quality. Positive ideas about metrics to improve testing and consistency are - a sure bonus. You should resist and fiercely oppose any attempts to link automation and testing, claiming that automation can and should work independently.

Second principle – “Definition and Meaning of testing cycle”: Never challenge or probe definitions and meanings of word “Testing cycle”. Keep it (testing cycle time) loosely defined so that you can flip in any direction when confronted by a skeptic challenging your claim of cycle time reduction. The more the vagueness of the term “testing cycle and cycle time”, higher are the chances of meeting goal of reduction of cycle time through automation. Any variables and factors that make “testing cycle” somewhat unclear – should be ignored and not discussed. It is important to have faith in the fact that “testing cycle time” is universally known term and does not need to be redefined in any context. You would be looked upon as genius if you simply talk about “cycle time reduction” and omit the unnecessary qualifier “testing” or “test” (as in “test cycle time”).

Third principle - “Playing to the gallery”: Use words like “business needs”, “business outcome”, “business processes”, “success through business alignment”, “Time to market” etc., liberally during all communications related automation and testing cycles. Confront any opposition by skeptics about automation and its connection to cycle time by “calling authority” – say “Business/Market needs it”. Strongly believe in statements like “Automation will help releasing products and services faster – and hence will improve customer satisfaction and company bottom-line”. Your success in achieving cycle time reduction depends upon how often and how strongly you make reference to “business” and “business needs”. These powerful words that do all the magic required. You need right rhetoric to spread the message. Another important keyword here is “market”. Make sure you thoroughly mix up and use the terms “Testing cycle time” and “Time to market” interchangeably. The more you talk about “Time to market” and how automation can directly help it – more you look authentic and convincing.

Forth principle - “Motivations and Incentives”: Believe economists when they say “incentives” drive change and motivate people. How can this not work here? The last but not least of the measure that one needs to be aware of is to provide incentives for people to reduce cycle time through automation. Define performance goals of individuals involved in the automation in terms of cycle time they reduce through automation. Penalize those that question the idea and fail to meet the performance goals. The automation initiatives tend to be successful in their stated goals of cycle time reduction when they are integrated with performance goals of individuals involved in the game – especially automation team. Also make sure (when automation is done by a decentralized team) to define and impose the performance goals ONLY on automation team and suppress any attempts to include manual testing team. After all automation and testing are not related in anyway and it is the responsibility of automation team to bring about cycle time reduction. Right?

If you find these principles and ideas rather strange and are intrigued – one of the several possibilities is that you might be working in a software product organization as opposed to IT and IT services organization. The folks in software product organizations that build software – unfortunately approach automation some what “differently” and cycle time might not be a very familiar term to you.

If you are not from software Product Company and still confused about the ideas in post and you think they are inconsistent or incoherent with each other – do comment. That is a good sign that you are thinking about the topic.

Let me know if there are other ideas and principles that you might have successfully used to reap benefits of cycle time reduction through (and only through) automation – I would be more than happy to incorporate them with due credit.

Shrini

All wannabe software testers out there …

An anonymous comment posted on my blog read “hi i am non IT background i want to do software testing course in india. if some give me some of the institution in india who teach very well that help u when u get job for software testing and ISTQB test exam.”

Nothing new here, I receive mails regularly asking for suggestion on how to get on a fast track for software testing and get a job. This particular post has three dangerous things that I see – which a new entrant in software testing should be aware of and avoid them.

“Software Testing course” – Let me tell you from my experience – there no such course in world that can make you a tester worth a job overnight. Any course that claims to do is a complete hoax and fraud. Shorter the course duration and taller the claims made by it – deadlier it is. Folks – be aware of such courses that claim you to get a testers job – please don’t fall into the trap. Another dangerous mix or variation here is – claim of “teaching automation or one or more world leading automation tool”. If you want to be software tester - no matter whether you are from IT or non IT background – don’t waste your money and/or time on such courses.

“ISTQB” (replace this with any popular testing certification” – while much has been written by my colleagues and many real life (not so pleasant) experiences out there – I would want to touch upon one thing. ISTQB or for that matter any testing certification WILL NOT teach you how to DO testing and how to gain expertise at it. That is their limitation. Please understand it. Considering certifications as businesses making money – teaching testing and assessing testing skill of a candidate in real time is not their cup of tea. Real certifications and exams that do subjective assessment of testing skill – are not scalable and hence certification people can’t make fast money.

ISTQB and other certifications have done one thing well – marketing. In India, many organizations and recruiters insist one or other testing certification as a must for an entry level testing job. In some others, attaining certifications is a criterion for promotion. Sad state of affairs – though. Sad – because it sets a wrong precedent. It creates wrong expectations among entrants and companies that hire these newbies. It creates a wrong image about software testing in general and at times it trivializes the craft. I have people telling me how easy was to pass a certification exam when they had no prior background or practice about testing. At the most you can get to know some terms used in testing and their fixed meaning as used by a specific group of people. Worst - in some cases they are taught as though those terms have universal meaning and acceptance.

Getting Job – this is third danger in the wannabe’s should avoid. While in some cases you might be lucky get a job using some testing course or certification (hiring process for software testing in India – lot needs to improve – but that is a different topic) – you will not survive long unless you practice real testing – get your hands dirty, feel and learn to think like tester. I can relate software testing skill to that of a musician. If you want to be an expert guitarist – what would you do? Take a 2 week course and a certification (theory) exam and claim a professional guitarist job? As taking a 2 week guitar crash course will not make you a competent – taking a software testing course will not prepare you for a professional career. Similarly learning few words and vocabulary related to music and guitar – will not help you to give performances – though knowing common words and their meanings can be helpful. This is not much different with respect to software testing – getting an ISTQB can help you to know words like “regression testing” and “severity of a software bug” – but those are mere words. Role of ISTQB ends there and real work of practicing testing starts.

In a nutshell – if you are reading this post, is someone who is trying to make it to software testing, want to get an IT job in software testing – here is what you need to keep in mind

Don’t do these things:

1. Look for or ask for software testing institute that gives a software testing course (shorter the better. One that guarantees job – the gem)
2. Taking a certification – especially when you have no clue on what is software testing.
3. Look for job on the basis of 1 and 2 above.
4. Don’t take a crash course on automation or automation tools in order to get a software testing job.
5. Take shortcuts

Do these things (few of things that personally helped me in my career) – Try them

1. Have a time horizon of 1-2 years to the minimum and a complete dedication to learn and practice software testing
2. Get a mentor – there are many willing to mentor if you show real passion and dedication. Read software testing blogs and engage in conversation. Use social media to your advantage – blogs, twitter, facebook, linkedIn. Make your presence felt as hungry, passionate new comer in software testing – let world notice you. Build reputation.
3. Practice testing - A platform I recommend is weekendtesting.

Any questions?

Shrini

How IT deals with Test Automation ...

Here is a short post on how test automation is dealt in IT/IT services organizations.

For example : A typical test cycle:

Before Automation -- (AS IS statistics - documented)

Number of Manual test cases = 300

Time taken to execute these test cases = 10 Person days

Tester's productivity = 30 Test cases per day (derived data)

After Automation -- (Assume 150 of these test cases are automated)

How do you think test cycle would look like?

Number of test cases Automated = 150

Number of test cases requiring manual effort = 150

Time taken for 300 test cases = Time for Automated execution + Time for Manual test execution

= (how much time will 150 automated test cases will take to complete) + 5 days (with 30 test cases per day productivity)

= 1 + 5 (assuming that automated tests run at 1/5th of time of manual execution)

= 6 days

Hence a business case will be developed to justify automation investment

The business case would say - with automation, we can save 4 person days of testing effort per cycle. Depending upon how many cycles of testing are lined up - we can break even the automation investment.

Oh... What a great feeling ... This is what I see all day in and out..

Let me tell you one thing .. executives, managers love this stuff ... so neat - objectively stated in numbers, justifying in terms of monetary terms why we need automation.

But some discomfort and terrible feeling in me ... what you people think - we are missing here?

Programmers make Excellent Testers - Arguments and Counter Arguments

Janet Gregory’s post on Programmers as testers prompted me to do this post. She mentions in the post that “Programmers make excellent testers”. So I asked myself can I think of few reasons to support and few others reasons to refute the statement. Here I go …

Programmers make excellent testers because:

1. Programmers understand their code and their fellow programmer’s code /very/ well. Hence knowledge of the code helps them to test /it/ better. Creator knows the best about his/her creation having created it.
2. Programmers understand /better/ the technicalities of the platform (anything other than “software application under test”) on which their code runs.
3. Programmers can /write/ /better/ automation code that tests their product code. Writing automation is part of testing right? Or test driven development?
4. Programmers being closest to the code – can find and fix the bug in smallest time possible – hence can do efficient testing. First opportunity of finding bug in a code is with programmers. If there is a problem in the code – they are the ones that should know about it FIRST.
5. … I can’t think 5^th one … probably need some help...

Now, let me cross over to other side.

Programmers make not-so-good testers because:

1. Programmers are /usually/ blind to their own mistakes. Their testing is limited by cognitive bias (confirmation bias)
2. Programmers /typically/ good at “construction” work – getting spec to working code – not a key tester skill. Programmer testing is more like a cook tasting food made by him/her before serving.
3. Programmers testing is /typically/ happy path – where would they get time to do “out of the box” as testing, unusual paths, usability, security and performance related tests? (Unless explicitly called out as a part of specification). Programmers /often/ do not see big picture.
4. Programmers, all through their professional life, work to improve their coding skills – so testing is a part time work for them (which programmer would work to improve testing skills unless he/she decide to become full-time testers)
5. It is hard for programmers to /think/ like users (many types of them) – their mission of Spec2Code limits them to think in terms of code

A bonus: Typically programmers hate or avoid testing (other than writing automation –which is again a coding work) as far as they can. Many would say “testing is not cup of tea” but I need to do it as we all in a team are responsible for quality. Programmers can’t make excellent testers simply it is not their job (in all).

In sport like Cricket – there are specialist batsmen, bowlers and also all-rounders (who can do both equally well). That is not true for testing.

Someone might say Janet’s context is /Agile/ development model – well, how does that change my “for” and “against” arguments here? How far does that matter?

Update : Pete Houghton (Twitter @pete_houghton) mentioned that debugging is a form of testing - programmers do it well. To me, debugging is an act of hunting down a reported or known problem and fixing it. Thus debugging follows testing - not a form of the latter. By going through the process of debugging repeatedly, programmers gain understanding of how they (programmers) make mistakes and how to avoid them. That also helps them to think about interesting ideas about testing. That is one reason that helps them to be better testers.

BTW, working in customer support - gives probably the best experience for testers, through exposure to wide variety of usage patterns (many of them - out of scope of specifications). Unfortunately - end users do not use applications as per the specification or user guide.

Shrini

Book Review: Selenium 1.0 Testing Tools - Part 1

In a first of its kind request, few months back someone requested me to do a book review – that too a book on “Selenium”. I wanted to learn Selenium since Long time, to compare and contrast traditional GUI based automation that I primarily worked on.

That is how I picked up this book by David Burns – a senior software engineer in test at Mozilla Corporation (Not Mozilla Foundation) I am not an expert in Selenium – this review of mine as from the eyes of a learner. I have attempted to convey how someone like me trying to learn Selenium would find this book. Those who are with Agile movement and likes might find my comments rather unusual – bear with me. I am from a different world …

Where do I start from? Before I started reading the book – I did some reading on the web. I was looking for a good definition for Selenium. Adam Goucher defined it as “Selenium is, at its core, a set of tools which let you control a browser. What you do with that control is completely up to you. Some frequent fliers use it to reserve aisle seats on their flights, but the majority use it as part of their testing process. After all, the end user of your product is not going to be experiencing the product through their browser as well, not via some wire-line unit testing framework.” I think, Adam’s definition is simply the best one to get started.

I would strongly recommend anyone starting with Selenium to read its history and especially support from Google.

As it happens with all most all books on “selenium” – even this book claims that Selenium tests the application and not selenium is a test automation framework. To me “testing” and “automation” are two different but complementary things and I think Selenium as an Automation tool rather than a “testing” framework.

Here is an account of my experience with the book – chapter by chapter.

- The book starts off with IDE installation and Selenium- “Hello World” kind of test (record-playback). The chapter covers few additional items to sample tests such as adding comments, debugging tests, working with multiple windows. AJAX applications make a sudden appearance in the chapter - not sure why. I feel the section on AJAX is not well connected with rest of the chapter. The section on “Saving Tests” is very brief – could have been expanded to cover exporting or converting IDE tests in other languages (Java) and automation frameworks. A brief introduction of TDD or ATDD or kind of automation that Selenium supports would have been a good way to start a book. Explanation of what is “Selenium” lacks depth and can confuse beginner and experience a like.

- The chapter on “locators” helps the reader to learn about locators. Coverage on DOM, XPath, CSS is pretty good. I would have liked the introduction be deeper. Finding the elements that make up a webpage is the work of these locators. You can locate elements by ID, name, DOM (through java script), XPath and CSS selectors. If you read the introduction of the chapter and summary – I am still confused about this question – “When we can get the page elements through record and playback function – why one needs to bother about locators – so many of them”. I think I know the answer. But I would have expected the explanation to be part of the chapter… “Why one needs to learn about Locators”. If you don’t ask this question – you can very easily follow the chapter and do hands on exercises without any problem and learn about IDE and some tricks

- As it is the case with other chapters so far – the introduction of chapter “Pattern Matching” lacked the depth and connection. It does not seem introduce the chapter by attempting to answer “Why pattern matching? Where? How this is applicable to automation in selenium”. The chapter, as in others mentions “In this chapter we shall do …..” I would have expected the book to cover – why part of these activities. I would have liked to see “how these activities fit in overall landscape”. Usage of pattern matching in general has been explained well. Use of “exact”, “glob” and other nuances of regular expressions, is covered in detail.

I will cover in next posts remain chapters of the book … Do Check back

A Paradox of Reification – A Tool or a Fallacy?

Reification – Making some idea into a thing [Wikipedia]

Reification is a process (or is an essential element) of modeling – a process of reducing a problem involving multiple variables into single or few variables (For example say “drinking some brand of health drink makes you grow high or makes super intelligent”). If you see a complex (say meaning something that involves many interacting variables) problem made simple (reduced to a single variable problem), be sure to find “reification” hiding around. I would say it is a form of “reductionism”

Strangely enough, the life of Sales/marketing professionals, I think, involves nothing but reification. They can’t sell anything if they become aware of reification and start thinking about model vs. reality. A salesman selling insurance policy would say “Buy this policy – your health is secured – health is reified in terms of money you will get to pay for illness as and when it happens.

Politicians are another group of people who live by reification (I am not sure how many of them are aware of this term, though they might be aware of its effects). A politician is constantly required to create simple models of social life and impose them on his target electorate so that she can win votes. How else you think one can make promises to bring social equality, eradication of poverty by distributing rice at 2 Rupee per Kilogram or provide 70% or so reservations to certain class of people? It is reification @ work. Can any politician dare to think about (giving up) reification fallacy?

See the paradox in the act of reification. As a manager you need to support (and possibly create many new ones) reifications like “Continuous improvement”, “Customer focus (yes this is a reification), “SMART goals”, “Consistent Processes”, “Best practices”, “Metrics”, “objectivity”, “Predictability”, “knowledge transfer”, “business value”, “customer satisfaction”, “transformation” and so on. But when you become the target (rather a victim) of such reifications – you suddenly become aware of the problems of reification – while you may not know or identify by this name or label. How many of us and how often we disagree on appraisal ration ratings given by our managers (yes – your appraisal ratings are biggest and most influential reifications in our professional/corporate life).

Try asking a director, CTO, Head of Testing of a software organization – what is testing? He is most likely to talk about best practices, consistent process, business transformation, customer focus, continuous improvement etc. This person needs to reify ideas and abstractions so that he can do his job. He would not let complexities at the deep-down disturb him and simple picture is what he needs to work with.

Now, note that reification is not simplification. It is a process of abusing “simplification”, resistance to think beyond the world of abstractions. Thus, depending upon which side of the fence you find yourself in most of time, reification becomes a tool or a problem or fallacy.

In short the paradox is – you can use reification to work (mostly managers and people who need work with simple abstractions) as a tool. When it bites back to you as a victim (as in appraisals) – you would hate it and fight it as a type of “logical fallacy”.

Why wouldn’t Sales/Marketing folks think about reification as a fallacy? Do they?

In Pursuit of Quality, let’s Call Quality as something else

The theme of this year’s Euro STAR conference is “In Pursuit of Quality”. A great theme but an ambiguous I would say. I wonder what could have been the meaning/interpretation of the phrase “Quality” here. One might say “your own definition of quality” – in a group, it is a problematic one with the missing context.

It is interesting to note many in software profession acknowledge that the phrase “quality” has many meanings – different people at different instants of time paint/experience quality differently. Many admit that objectification of “quality” should be avoided. Yet, it is a rampant practice to portray “quality” as having some universal objective meaning that everyone agrees. Popular uses of the phrases like “Quality Software”, “Cost of Quality”, “Quality Engineering” etc., are examples of such practice. For many, “quality” is just a bucket where everything related to expectations of the users can be lumped together. It is one term that evokes a great array of human emotions from extreme pleasure to awe, from frustration to sense of achievement.

Let us imagine “quality” in terms of how Quantum Mechanics describes quantum states of subatomic particles. This “quality” is a like a subatomic particle in quantum superposition state and represents many meanings (or states) at any given context (or Einstein’s space-time). When one makes a swipe at this term and uses it with respect to a context saying “this is quality stuff” or “this quality is unaffordable” or “quality is free” – the wave function collapses and a particular meaning of quality shows up.

As Michael Bolton explains while posing a testing challenge about agile teams -

”For example, in a particular context, if we want “quality” to mean “bug prevention,” let’s say precisely that. Then let’s recognize the ways in which certain approaches towards preventing bugs might represent a threat to someone’s current interests or to their personal safety rules. If, in a particular context, we want quality to mean “problems solved for customers”, let’s say precisely that. Then let’s recognize that there are many approaches to solving problems, and that some problems might be solved by writing less software, not more. If we want “quality” to mean “many features in a product”, let’s say precisely that. Then let’s recognize how “many features” can satisfy some people—while adding complexity, development time, and expense to a product, thereby confusing and annoying other people. In other words, let’s use the word “quality” in a more careful way, which starts with deciding whether we want to use the word at all.”

That is to say if you want to say something to indicate quality – say it (without using the term quality). In this way, you can prevent misuse of the term for the better. If you want to pursue quality - say what you exactly mean so that people can understand you without confusion … Probably people will stop using phrases like “Cost of quality” or “Quality plan” or “Quality Assurance” or stop using quality with another set of abstractions like “Creating business value through quality software”

To repeat Michael “Quality isn’t a thing, but rather a complex set of relationships between products, people, and systems. We should be calling quality as something else that states what it is in a context rather than calling it with one word always.

I have a suggestion. Instead of using QUALITY – let us use words made of up characters in the word “QUALITY. For example - you might say LITYQUA of this software is poor to indicate usability or you might say TUAQLI of this software needs to be worked on to indicate security or cost. Once thing is sure to happen when you do this – people will say – what? When you explain them the meaning or interpretation of attribute that you are trying to represent is using the “jumbled” up word – they might relate it to that one thing and only that thing. Thus misusing and lumping “all” in to Quality, stops – I hope.
Try saying some other term when you want to say “quality” – difficult?

Two versions of being practical...

My boss tells me always that my ideas about testing are impractical and esoteric. I find it hard to understand why he thinks so. I believe my testing and automation ideas are as practical as one can get because I can demonstrate the practicality of the ideas by "doing" testing or automation. I suppose being practical - should be mean "demonstrable". I can show how my ideas of testing can be and are practical. Boss is not impressed.

Probably, my boss has another version of practicality. It goes something like this. Regardless of what subject matter or technicality of testing as it is practiced in real time - there is a social, political, business side to it. Many things that testing practitioner believes to be true are not possible in real world due to political and non technical aspects around testing. Under such situations - being practical means changing your testing philosophy and approach to suite the context of the testing world. Harsh word for such adaptability could be "compromise". If you are surprised or questioned instances like stakeholder seeking quick benefits of automation (something that reduces cost of testing), counting bugs, counting requirements, using bug metrics to measure testers, expecting testers to find all bugs, doing 100% testing - you probably are practical in another sense.

Living with many irrational notions about testing and trying to do what is possible, taking the world as it comes without challenging it or fighting the problems is the version of practicality - probably my boss believes.

Both of us could be right or wrong. How do I know?

This reminds of me of a Jerry Weinberg's saying (para phrase) - "Instead of calling something as irrational or illogical - consider it as rational or logical from another perspective or set of values".

You would like to call "testing" as an engineering pursuit? Wrong... it is becoming (it was from the beginning but now it is showing up more clearly) more social. Testing works in social systems and is affected by cultural, emotional, economical issues.

Time to start studying economics, social science to reason behavior of people in testing context. It is not rational at all times....

Shrini

What type of tester are you?

I was assisting homework of my 7 yr old daughter in her science assignment. The assignment was to collect pictures of living and non-living things and make a collage. I thought about how one could approach this work and have fun while doing it. Being a tester, it is hard not to see connection in a work like this. Here is what I thought….

Let us say two testers “A” and “B” are given this homework assignment of creating collage of living and non living things.

Tester A: Living and non living things? Just collect few pictures of animals, plants, humans - paste them as living things and collect pictures of cars, buildings, mountains, bridges and paste them as non-living things. Home work done. Test pass and on to next thing…

Tester B: While thinking about traditional ways of looking at living and non living things, this tester also questions his own understanding and meaning of “living” and “non living”. He looks around, reads, discusses with colleagues about these things. He, then ends up with a broader list and accompanying narration.

What is the difference between these ways of approaching a task at hand? Tester A is more focused on “completing” task with minimal mental engagement about the settings of the task. He takes “widely accepted” meanings and understanding of the things and optimizes the details of the task. Tester B, starts with questioning the premise of the task, meaning and understanding of the things required to complete the task. He seeks a broader understanding of task elements and attempts to question biases, prejudices of his own and others around. Though on the face of it, it might appear that Tester B will take longer time to complete the task (homework), through practice, he might able to finish the task nearly at the same time while understanding of broader aspects of the task and clarifying biases and assumptions.

Today’s testing industry promotes type A testing mentality. Type A testers eventually will become “commodity” (with price tag 20 Dollars per hour – can execute 30 test cases per hour etc). I am not sure that is a good thing or not. If you are a manager or someone who manages testing as business – you would probably have (or would like to have) more of Type A testers as such people are plenty. Simply, take truck load of freshers from college, run them through 15 days of training on testing vocabulary, make them memorize some domain stuff (banking, insurance, telecom etc) , teach QTP – there you go an army of (Type A) testers ready for deployment (hence billing).

If you are a tester looking at long term fulfilling career – you would want to take path B – practice to do things differently, challenging status-quo, thinking broadly about the problem and its context etc. To become “Type B” tester you need practice – practice of doing testing (weekend testing is an excellent platform or training ground). I am afraid there is no shortcut here – no 15 day testing courses to get there.

There is a catch here… Type B testers are mostly rebels and free thinkers and often want break status quo, create new paths, explore new territories. Much of our industry is not very nice with such folks. Very few can tolerate “ever questioning”, skeptical and rebellious tester in the team. In a control freak, military kind of set up where key focus of the job is to “follow instructions” or “getting job done” – Type B testers get frustrated. Some get converted into Type A or leave the organization. Sad situation but true in many cases. I hope that one day either rebel testers will learn to live harmoniously in a hostile, confirmatory environment or industry recognizes the value of these “problematic” folks and make space for them.

“Testers are like lighthouses at sea shores, headlights of an automobile” says James Bach (paraphrase). Good testers, apart from completing the task given to them, engage in parallel about “meta” things for better and deeper learning. That is a good thing (can I say for testers not for folks who manage testing as a business?).

Did someone say “Type C” tester?

What does that mean?

The other day I visited a science exhibition at my daughter’s school. As I was passing through an array of models and demonstrations, I could see every student was enthusiastically trying to draw every visitor's attention to his/her piece of work. Everything sounded good except one thing - the students were seen literally reading some sort of script to explain their work. Some smart guys memorized the stuff well so they did not require any hand written (mostly) note to assist them delivering the script but many did keep one by their side. It was hard to find someone who “knows” what he/she is trying to explain rather than forcing the stuff through using pale dialogue delivery.

There at the corner, a bright spectacles boy waved his hand calling me to see his work. I walked up to him. He was demonstrating “law of conservation of momentum” through a model that roughly looked like the one above.

After a customary greeting, he read through his memorized statements about law and looked at me with hopeful eyes asking if I understood anything out of it.

I asked him “what does this law tell you”.

He promptly, this time with more energy and desperately attempting to sound confident, repeated the law (the statements) like a tape recorder.

What do you understand from the term “momentum”, I asked him.

He was totally unprepared for the question (missing in his notes - missing test case?)

He quickly recovered gently tapping his spectacles said “It is simple sir - it is mass times the velocity”

I know that is a mathematical formula for momentum.

Student finally found something that I agree with him and said “Yes. you are right sir”.

At that instant, I remember my hero Richard P Feynman - the (most) curious character and asked him “But what does that mean? Mass times velocity?”

This time, student was not expecting that question at all.

I moved to another demonstration where they were showing double helix model of DNA and the girl explaining the model, gave a 5 minute non stop explanation of what DNA is and what genes are.

Again, some question “What does that mean” to the explanation “genes carry genetic code so that characteristics are passed on from species to species” got everything to a grinding halt.

I became pretty notorious from that day in among my daughter's friends about asking such silly questions about things that are taken for granted for the students. My daughter later told me that her friends were impressed with my “latest” knowledge of science and my questions.

Later, over the week end, one the students (harassed by me in the exhibition) paid a visit to my home apparently (I thought) to take revenge on me. He, incidentally was my daughters classmate and asked me “what do I do for living”. I said to him “I am a software tester”. While he did not understand that very much, he wanted to know how learn things. I explained him about few things that I learned of late as a student of software testing (than when I really a student to learn science). He appeared to have understood some of my approaches.

When he was about to leave, I gave this book “ Surely You're joking Mr Feynman” and told him - "learn science like this guy". I was happy that I introduced Feynman to a student hoping the “Feynmanism” spreads in the school.

I wish a very happy and prosperous New year to you all

Shrini