Learning from Tenali Raman's crows ...

As kids, like many in southern part of India - I grew up listening to stories of Tenali Raman - a 16th century wise court-poet of King Krishnadevaray of vijaynagar empire. Tenali Raman is also known as Vikat Kavi - meaning intelligent poet. Birbal from King Akbar's court - enjoys similar cult among kid's stories in India. This story of counting crows that I narrated to my 8 year old daughter - made me realize how real are Tenali raman's crows in our day-today life in software.

First, let me quickly run through the story. One day king throws up a strange puzzle to Tenali - asking him to count and report the number of crows in the city. Tenali thinks for a while and asks for 2 days time to come up with the answer. After two days, he comes back and reports to king that that there are One lach (10 lach = 1 million) seventy thousand and thirty three crows in the city. At first, the king becomes frozen and did not know how to respond - after a while, recovering from the shock of the answer - king checks if  Tenali is sure about the answer. King further says that he would conduct a counting (recounting?) and if number does not agree with Tenali's number - he (Tenali) would punished. Tenali being Tenali - responds to qualify his answer. He says it is possible that the recounted number of crows might be different from his number. If the new number is less than old number - then it is due to the fact that few of city's crows have gone out of station (city) to visit their relatives in nearby cities. If the new number is more than the old number, then additional number is due to crows from nearby cities visiting their relatives in vijaynagar city. Listening to this - king has a heart-full laugh and realizes the flaw in assignment/problem. As it happens in all Tenali stories - Tenali gets king's praise and some prizes for the witty answer.

Now, let us come back and see how this crow metaphor is applicable to what we do as project managers, test managers and testers in our day today work.

There are entities we deal that are similar to crows - in following respects :

1. Counting/quantifying is a prized puzzle
2. Counting is asked by an authority, a boss - you cannot say "No" to ( saying "no" can cost you your job or potential label of "incompetent")
3. Often you can fake a number
4. There is no easy, sure way to verify/validate the count
5. Even if someone does a recount and comes up with new (different) count - you can always "explain" the discrepancy, like Tenali did.

One example that comes to my mind is count of test cases. Typically, during test estimation process, as a test manager you would be asked "how many test cases could be written for a given set of requirements". The boss would then do the required math to confirm the number of testers required, time required to execute the estimated number of test cases (note - time required to "execute" test cases - not to test). So, wear hat of Tenali - throw up a number. If asked - show your working (be sure to have your working).  You would be OK from then on.

There are things we deal in software that can not be counted like we count concrete things.  Software requirements, use cases, test cases, lines of code, bugs, ROI from Automation - are abstracts not concrete objects. Counting them is akin to counting crows as in Tenali's story.

[Puzzle : Prove that ROI from automation is a Tenali Raman Crow count]

Cem Kaner says executives are entitled and empowered to chose their metrics. So, King was perfectly right in asking Tenali to count and report number of crows - though objective of King in the story is not to make any important decision for his kingdom. In any case - crow count metric was sought.

What can a tester/test manager do when asked to count "crows" ? While our community develops models and better alternatives to "imperfect metrics" - we need to tread a careful path. We should provide alternatives and possible answers to crow counts.

I have come to realize that refusal to give the count might be counter productive in many cases - trying to ape Tenali Raman might useful. Need for quantification is here to stay - years of persuasion and reasoning why in some cases counting can be bad - has not managed contain the problem.

What do you think about "Pass/Fail Counts"?

Shrini

My Views on Testing certification : 2012

A reader of my blog "Arpan Sharma" writes "What’s your take on certifications these days? I see your wrote about this is 2008 which is almost 4 years ago. Do you think the landscape of certifications have changed in recent times?".

Arpan - Thanks for writing and reminding that my stand on certification on this blog is about 4 years old now. It is interesting that you are checking with me if I have changed views. Here is how I summarize my current thinking on certification.


1. First of all the person seeking certification should be absolutely clear what they are expecting the certification to give them - Knowledge, Skill, skill enhancement, Marketing value, a job, an interview

2. Certifications that do not observe and qualitatively grade a tester - in action "while doing testing" - can not guarantee a certain level of skill in testing. Employers, Recruiters, hiring managers - please take a note.

3. If you want to learn how to do good testing, how to gain skills in broad testing landscape - certification is not something you should look for.

4. If there is a certification that let us get a job in a given situation/context or gets you a interview shortlisting - you should consider taking that certification. But - be aware - once you get your job - you are on your own. You would then be required to display (depending on the type of organization and nature of job) skills on job. Certifications' role ceases there.

5. Be critical about certification material and tests tell you - question them. Form your own ideas and logic about how things work. Do not take everything that taught or you read as part of certification as "universal truth. Why this is important? Only being critical on what is certification course - can help you to decide what value intrinsically you gained from it and what already existed in you.

6. Reputation is everything in today's world. You gain professional reputation by demonstrating your work and skills to your employer and to out side world (through networking world). Building reputation takes time and real good work. People with confidence in their skills and reputation - do not require a third party to endorse their level of skill. In today's world - people with skill and reputation - don't need certification. What does that tell you about certification?

7. Take special note of qualifiers like "Advanced" when applied to certifications - check out what is advanced and how? More often that not - it is more "jargon-laden".

#4 and #5 specially apply to freshers looking for /some/ job and those 1-3 years experience folks who either had some software job or a lost a testing job.

In terms of landscape of certifications - I don't think there has been change. Prime motive for certification providers is to make money - fast and cheaper. That has only intensified with many job seekers. That is fine as a business objective - we the target audience of such business ventures need to be clear about what we want from certifications and how capable are these certifications to deliver on the promises.

I repeat what I said earlier - if you want to learn, acquire skills, enhance skills in testing - certifications are the things that you should avoid. There are better, cheaper ways of doing that.

Did I answer your question Arpan?

Shrini

Patterns in weakness in approaches about testing

I was reading this testing round table discussion and thought this might a blog post. Here I go...

To me, the biggest weakness is the perception or idea about what testing is  and why it is required.

Here are few examples as how companies treat testing.

1. Something that is avoidable to large extent or even can be eliminated if they could get their programmers and analysts get spec and code exactly right. The lousy work that these folks do during SDLC - creates need for testing.

2. Quality Assurance - Straight out of the box comparison with manufacturing assembly line. For these folks, testing is all about process and nothing else. If you get process right - you are done. It does not make any difference who does and when - all they need is to get process script right.

3. Building quality from the grounds up - A variation of #1 above - a growing group of people think that if you have automated tests (checks - actually) you don't have to really worry about testing. You are building quality from the ground - you cannot test quality but need to build it right? poor testers will not define and manage testing (under the name QA)

4. Testing? What? The whole team approach. This is creation of Agile model. Here, testing is everyone's responsibility. There goes "testing" out of the door as a specialist's job. When testing is something anyone in the team does - it is like any other project task.

5. Dont forget - this popular rhetoric - Testing (phase) is dead - That is biggest weakness in approaches about testing.  What else can be the biggest weakness about something other than saying it is dead?


Whole idea of testing is dead is around these beliefs and notions (test yourself if you agree or not)


1.  Testing (phase or role) makes developers complacent - a safety-net - remove it to make developers responsible.
2.  with so much focus on automated unit testing, test driven development, continuous integration - developers are producing quality software anyway
3. Finding problems is no big deal, we know where problems are (this is what James Whittaker said in his EURO STAR 2011 keynote). So do we need what testers for?
4. With cloud as popular software delivery model - you don't bother about bugs leaking. Time and effort to fix and turn around bug is ridiculously LOW - why bother testing?
5. What for you have crowd sourcing? Beta testing? - throw your stuff to users - let them use and tell us where are the bugs (there should not be many as we are group of smart developers and we know where the bugs are)



Thus,  the weakness in testing arises out of how we think about it  and what we want it to do for us. Thinking idealistically about how software is made and used, applying models from other fields without properly customizing them and removing or de-emphasizing the human element in the system  - are the key patterns of weakness in testing approaches




What do you people think?


Shrini