Two versions of being practical...

My boss tells me always that my ideas about testing are impractical and esoteric. I find it hard to understand why he thinks so. I believe my testing and automation ideas are as practical as one can get because I can demonstrate the practicality of the ideas by "doing" testing or automation. I suppose being practical - should be mean "demonstrable". I can show how my ideas of testing can be and are practical. Boss is not impressed.

Probably, my boss has another version of practicality. It goes something like this. Regardless of what subject matter or technicality of testing as it is practiced in real time - there is a social, political, business side to it. Many things that testing practitioner believes to be true are not possible in real world due to political and non technical aspects around testing. Under such situations - being practical means changing your testing philosophy and approach to suite the context of the testing world. Harsh word for such adaptability could be "compromise". If you are surprised or questioned instances like stakeholder seeking quick benefits of automation (something that reduces cost of testing), counting bugs, counting requirements, using bug metrics to measure testers, expecting testers to find all bugs, doing 100% testing - you probably are practical in another sense.

Living with many irrational notions about testing and trying to do what is possible, taking the world as it comes without challenging it or fighting the problems is the version of practicality - probably my boss believes.

Both of us could be right or wrong. How do I know?

This reminds of me of a Jerry Weinberg's saying (para phrase) - "Instead of calling something as irrational or illogical - consider it as rational or logical from another perspective or set of values".

You would like to call "testing" as an engineering pursuit? Wrong... it is becoming (it was from the beginning but now it is showing up more clearly) more social. Testing works in social systems and is affected by cultural, emotional, economical issues.

Time to start studying economics, social science to reason behavior of people in testing context. It is not rational at all times....

Shrini

What type of tester are you?

I was assisting homework of my 7 yr old daughter in her science assignment. The assignment was to collect pictures of living and non-living things and make a collage. I thought about how one could approach this work and have fun while doing it. Being a tester, it is hard not to see connection in a work like this. Here is what I thought….

Let us say two testers “A” and “B” are given this homework assignment of creating collage of living and non living things.

Tester A: Living and non living things? Just collect few pictures of animals, plants, humans - paste them as living things and collect pictures of cars, buildings, mountains, bridges and paste them as non-living things. Home work done. Test pass and on to next thing…

Tester B: While thinking about traditional ways of looking at living and non living things, this tester also questions his own understanding and meaning of “living” and “non living”. He looks around, reads, discusses with colleagues about these things. He, then ends up with a broader list and accompanying narration.

What is the difference between these ways of approaching a task at hand? Tester A is more focused on “completing” task with minimal mental engagement about the settings of the task. He takes “widely accepted” meanings and understanding of the things and optimizes the details of the task. Tester B, starts with questioning the premise of the task, meaning and understanding of the things required to complete the task. He seeks a broader understanding of task elements and attempts to question biases, prejudices of his own and others around. Though on the face of it, it might appear that Tester B will take longer time to complete the task (homework), through practice, he might able to finish the task nearly at the same time while understanding of broader aspects of the task and clarifying biases and assumptions.

Today’s testing industry promotes type A testing mentality. Type A testers eventually will become “commodity” (with price tag 20 Dollars per hour – can execute 30 test cases per hour etc). I am not sure that is a good thing or not. If you are a manager or someone who manages testing as business – you would probably have (or would like to have) more of Type A testers as such people are plenty. Simply, take truck load of freshers from college, run them through 15 days of training on testing vocabulary, make them memorize some domain stuff (banking, insurance, telecom etc) , teach QTP – there you go an army of (Type A) testers ready for deployment (hence billing).

If you are a tester looking at long term fulfilling career – you would want to take path B – practice to do things differently, challenging status-quo, thinking broadly about the problem and its context etc. To become “Type B” tester you need practice – practice of doing testing (weekend testing is an excellent platform or training ground). I am afraid there is no shortcut here – no 15 day testing courses to get there.

There is a catch here… Type B testers are mostly rebels and free thinkers and often want break status quo, create new paths, explore new territories. Much of our industry is not very nice with such folks. Very few can tolerate “ever questioning”, skeptical and rebellious tester in the team. In a control freak, military kind of set up where key focus of the job is to “follow instructions” or “getting job done” – Type B testers get frustrated. Some get converted into Type A or leave the organization. Sad situation but true in many cases. I hope that one day either rebel testers will learn to live harmoniously in a hostile, confirmatory environment or industry recognizes the value of these “problematic” folks and make space for them.

“Testers are like lighthouses at sea shores, headlights of an automobile” says James Bach (paraphrase). Good testers, apart from completing the task given to them, engage in parallel about “meta” things for better and deeper learning. That is a good thing (can I say for testers not for folks who manage testing as a business?).

Did someone say “Type C” tester?

What does that mean?

The other day I visited a science exhibition at my daughter’s school. As I was passing through an array of models and demonstrations, I could see every student was enthusiastically trying to draw every visitor's attention to his/her piece of work. Everything sounded good except one thing - the students were seen literally reading some sort of script to explain their work. Some smart guys memorized the stuff well so they did not require any hand written (mostly) note to assist them delivering the script but many did keep one by their side. It was hard to find someone who “knows” what he/she is trying to explain rather than forcing the stuff through using pale dialogue delivery.

There at the corner, a bright spectacles boy waved his hand calling me to see his work. I walked up to him. He was demonstrating “law of conservation of momentum” through a model that roughly looked like the one above.

After a customary greeting, he read through his memorized statements about law and looked at me with hopeful eyes asking if I understood anything out of it.

I asked him “what does this law tell you”.

He promptly, this time with more energy and desperately attempting to sound confident, repeated the law (the statements) like a tape recorder.

What do you understand from the term “momentum”, I asked him.

He was totally unprepared for the question (missing in his notes - missing test case?)

He quickly recovered gently tapping his spectacles said “It is simple sir - it is mass times the velocity”

I know that is a mathematical formula for momentum.

Student finally found something that I agree with him and said “Yes. you are right sir”.

At that instant, I remember my hero Richard P Feynman - the (most) curious character and asked him “But what does that mean? Mass times velocity?”

This time, student was not expecting that question at all.

I moved to another demonstration where they were showing double helix model of DNA and the girl explaining the model, gave a 5 minute non stop explanation of what DNA is and what genes are.

Again, some question “What does that mean” to the explanation “genes carry genetic code so that characteristics are passed on from species to species” got everything to a grinding halt.

I became pretty notorious from that day in among my daughter's friends about asking such silly questions about things that are taken for granted for the students. My daughter later told me that her friends were impressed with my “latest” knowledge of science and my questions.

Later, over the week end, one the students (harassed by me in the exhibition) paid a visit to my home apparently (I thought) to take revenge on me. He, incidentally was my daughters classmate and asked me “what do I do for living”. I said to him “I am a software tester”. While he did not understand that very much, he wanted to know how learn things. I explained him about few things that I learned of late as a student of software testing (than when I really a student to learn science). He appeared to have understood some of my approaches.

When he was about to leave, I gave this book “ Surely You're joking Mr Feynman” and told him - "learn science like this guy". I was happy that I introduced Feynman to a student hoping the “Feynmanism” spreads in the school.

I wish a very happy and prosperous New year to you all

Shrini

Fish baking story ...

This was a story that came to my mail box.

A little girl was watching her mother prepare a fish for dinner. Her mother cut the head and tail off the fish and then placed it into a baking pan. The little girl asked her mother why she cut the Shead and tail off the fish. Her mother thought for a while and then said, "I've always done it that way - that's how grandma did it." The girl was not satisfied with the answer, and went to visit her grandma to find out why she cut the head and tail off the fish before baking it. Grandma thought for a while and replied, "I don't know. My mother always did it that way." So the little girl and the grandma went to visit great grandma to find ask if she knew the answer. Great grandma thought for a while and said, “Because my Grandma told once that the baking pan was too small to fit in the whole fish”.

In most cases we fail to challenge belief systems and assumptions.

This is true about many ritual centric processes that we are asked to do in testing today. If you dig deeper, you would be surprised like this little girl. There are no real reasons to do certain things. Typical answers might be “my boss wants it that way”, “this is how our clients want” or “this is how I have done it always” or “this is in that book” or “this is how the consultant who we hired told us” or “this is how it always appear to work”.

One trademark quality of a skilled tester is skepticism and ability to think beyond rituals – ask “why I am doing what I am doing”.

Are you asking this question frequently in your job as tester?

Shrini

How can Mathematics be the Language of Nature?

I am reading this wonderful book about Physics (and history and philosophy of Physics) "Tao of Physics" by my all-time favorite Physicist and author Dr. Fritjof Capra, a contemporary of Werner Heisenberg. Incidentally, Dr. Capra's other famous book "Turning point" has really introduced me to the concept "systems thinking" while I struggled to grasp the subject by reading another influential legend and Guru of software (testing) Jerry Weinberg. James Bach and Michael Bolton introduced me to the subject of "systems thinking" as science of complexity.

Well... coming to the point, this post is not about system thinking but about a paragraph from the book "Tao of Physics". Here goes the text ...

"... The words of our language are thus not clearly defined... Science, on the other hand aims for clear definitions and unambiguous connections. Therefore, it abstracts the language further by limiting the meaning of its words and by standardizing its structure in accordance with the rules of logic. The ultimate abstraction takes place in mathematics where words are replaced by symbols... In this way scientists can condense information into one equation, into one single line of symbols. The view that mathematics is nothing but extremely abstracted and compressed language does not go unchallenged...mathematics not just a language to describe nature but is inherent in nature itself. Originator of this belief was Pythagoras, who made famous statement - "All things are numbers" …

This puzzles me ... considering(or conceding) that mathematics is an abstract language used by scientists to represent theories, models and scientific thought in a precise and unambiguous way, how can such an abstract language be suitable to describe nature (even in parts) ?

Nature is live, rich and multi dimensional where as mathematics is at other end of the spectrum - abstract, precise and single dimensional. Did Pythagoras and for that matter Galileo make mistake by saying "book of nature is written in the language of mathematics". I am certain that mathematics does not be to be interpreted where as nature and its manifestations need to be. Am I making sense?

Nature of Problem(s), we testers (should) investigate (or solve)

I am puzzled about the nature of problems that software testers, today are juggling with or should be losing their sleep over - more than the problems themselves. As humans, we love simplicity (not sure about about other species - not sure if there is any way to check if they love simplicity too). In every walk of life, we seek and embrace simplicity. Many experts tell the stories about how they have solved mysteries of the world through and give us the tools, techniques to conquer/dominate the world around us. We now have quantum computers even though there is barely anyone who has fully understood Quantum theory. We can make machines, apply our partial knowledge but make a mistake of claiming glory of scientific knowledge saying “this theory is true as I have experimentally verified it and there are practical applications”. The mistake that we make is about “completeness of knowledge” – making a machine based on a theory does not amount to complete understanding of the theory.


Alfred North Whitehead, the famous English mathematician and Philosopher warned us “Seek simplicity and learn to distrust it”. Simplicity is so tempting that it fools our minds and thinking and makes us glued to it. It appears to me that we have not learnt to distrust simplicity at all – worst many even does not know that we should “doubt” it. There is “banana” principle, attributed to Jerry Weinberg. For many things like simplicity, we have only learnt how to start but have horrible lost the trick “when to stop”.

Many in software world believe that software testers are expected to answer questions that, simply lead to “pass” or “fail”, “works” or “does not work”. These questions are like asking “is 4+4 =8? Is this an isosceles triangle? Is the temperature of this body 86 degree centigrade? What is National Stock market Index today?” Engineering, mathematical and scientific questions right?


In our software profession, we love to pose “engineering/scientific” questions that lead to binary answers. After all we are software ENGINEERS and software is ENGINEERED. Hence I started this post with simplicity and our obsession with simple models of complex things. What is the true nature of questions/problems that software testers investigate and solve? Let me make a bold conjecture here. I think they are similar to the questions that we pose in the realm of social sciences. For example, questions like “Is homosexuality linked to or mainly due to some mental illness?” or “Should burqa be banned in public places” or “Is there a God”. These questions are asked in a broad social setup and are linked various material practices of the people in the society, about their culture, history etc. Much in a similar way, software operates in a social context and people are mainly concerned about how software alters their day-today work. The nature of questions/problems requires non-binary, qualitative answers and solutions. The questions about quality are based on emotional, subjective feelings (there is no binary 0 or 1 about software quality).

As Michael Bolton says “decisions about quality are always emotional and political”. How can there be binary mathematics in emotions and politics? That is the point these software engineers and process people miss and insist on quantified definitions of quality and other abstract entities associated with Software. It appears to me that they are constantly conditioned by the business environment to believe so. Coming back to simplicity, software engineers (process/CMMI/Six Sigma) people definitely think that the problems that software testers/engineers solve (investigate- actually) are purely engineering/mathematical in nature. Hence they believe in absolute answers, causality, objectivity and certainty.


I think they are wrong !!!! When do we learn when to stop saying “banananananana..?” While you are reading this and interested in debating about “nature of testing problems”, many budding software testers are happily busy in churning out hundreds and thousands of test cases and blissfully shouting “pass” or “Fail”. Well, you might say do they have options … I am sure they do.

Shrini

A movement called weekend testing – What it can do to you and your testing career?

An extremely contiguous phenomenon is sweeping in some circles, communities of software testing; it is called Weekend testing (twitter @weekendtesting). Presumably, born out of few young minds (Ajay Balamurugadas, Parimala Shankariah, Sharath Byregowda, Manoj Nair ) and influenced by context driven software testing philosophy. In a short span of time, it has traveled from Bangalore (the home of Weekend testing) to Europe and the US. So it is globalized movement. As testimony of its popularity and the value, who's-who in global testing community have acknowledged it

It is hard for anyone who is a passionate/skilled software tester not to be associated with it. Once you are in, you keep going and loop in others. Such is the power of the concept and the way weekend testing works. I would probably equate (not mathematical sense) weekend testing to open source software development. What drives these movements? Passion, Desire to learn, Demonstrate skills in the real time "On Demand". The last point is something that is more apparent in weekend testing movement.

What makes weekend testing - a movement or a revolution in its infancy?

The objective:
Practice testing (or software testing) in real time with like minded (I wish if some dis-like minded folks join it – we need fight confirmation bias always as testers. Being vigilant about fallacies of "you will see what you want to see or you will fall in love with that which confirms what you know").

This objective is very powerful and has enormous potential to grow big. Keywords here are "practice" and "real time". Now you can see testers saying "I was practicing testing" just like a musician doing is "Riaz" or the sportsman sweating it out on the ground. A manager would call few of his low performing testers and say "Go and get some practice of testing in week end testing sessions". While his testers are training, the manager can check their progress by viewing the session transcripts and related blogs posted elsewhere. Transparency to the core!!!

The structure

Here is a generic format of how a weekend testing session is arranged – more detailed description can be found here

A group of testers sign-on to Skype or Google talk and get started with their testing at a specified time as announced on the weekend testing website. One of the testers assumes the role of facilitator giving the mission for the session and moderates the whole thing. Mission of the session is itself tested first – meaning it is scrutinized for possible ambiguities and a refined one is adopted. Testing starts bugs, issues are logged in an open source bug database and followed by a debrief session. The session transcript is posted with results for everyone to see and learn further. It is almost free for those that have computers and internet connection. Weekend testing thrives on usage of open source/free tools. It is more or less self-driven – all that was required to set up an initial structure for testers to get started and posting the results and transcripts of the sessions.

It is a terrific place to learn With so many willing, skilled and available testers and coaches around what you are learn is nearly unlimited. With a wide variety of products to test, variety of testing missions to try out – there is something new to learn in every session. With people across the globe participating, you get access to an interesting cross section of skills, cultural legacies to learn from. You can improve your communication, writing skills in a friendly yet mildly critical environment. People here tend to me cooperative and more interested in seeing everyone succeed. It is the genuine interest in other's growth and improvement that makes weekend testing a unique platform for greenhorns to gaze around.

It promotes "community" certification/acknowledge - not given by any "standards" like bodies. Needless to say the people associated with the movement are known for their "critical thinking" and would go to any extent to be "open for scrutiny/challenge any time anywhere". That makes this movement strong and really unbeatable.

It makes testing demonstrable in its deepest details possible. For long, people influenced by context driven testing have struggled to explain the power of exploratory testing. They have been trying to argue for skill in testing as opposed to so called best practices and process driven testing. And there "test cases" the ubiquitous building blocks of testing and the questions that were often asked "how can you do testing without test cases, How can you repeat testing? How will client accept that we tested?" Now we have a proof in a public movement and "open for scrutiny anyone anytime" – You can literally see testing happening in front of your eyes. What more evidence you need?

The People

The founding members of this movement are, at the core, very dedicated, passionate software testers whose sole distinguishing qualification is their appetite for learning and become skilled software testers. That drives them to do things that they do. They are constantly building their reputation through demonstrating their skill and learning constantly. So, hanging around with these people can be very dangerous. They will make you like them. The weekend community is very aptly mentored, coached and supported by people like Michael Bolton (no… not the singer or the "officespace" character), James Bach, Cem Kaner, Pradeep Soundararajan, Matt Heusser, Jerry Weinberg, just to name a few.

We (notice, I am using this "we" after a long hesitation as I have not contributed enough to include myself in this movement) are thankful to these leading lights for encouraging the movement and advertizing it.

The Future of weekend testing – The way I see it.

Few thoughts …

• There will be many chapters of weekend testing all over the world. There will be weekend testing conferences to the scale of STAR conferences in the US.
  
• Big universities teaching software engineering will recognize this movement and will have representatives from them
  
• Commercial product companies will engage weekend testing folks to test their products. With the support of crowd sourcing companies like utest, week end testing will gain authenticity and grow.
  
• Weekend testing will then get into weekdays as people want more of it – we then have to look for another name to call and identify ourselves … But that is an interesting problem to solve
  

I am looking forward to participate and contribute this movement …

Update (10th March 2010)

Parimala Shankariah has brought my notice that Santhosh Tuppad, yet another promising testing star in the indian software testing horizon, who was recently featured on utest - has been a very contributing member of this movement. Thanks Parimala for point this out ....

Santosh ... nice to know about your contributions.. Keep going... Look forward to participate in sessions with you.

Shrini

Man and the machine ....

Can a software be produced and maintained using a machines in a factory … like Google does?

Notice, with their machinery and factory setup, they have been able to cut down the testing cycle to 4 minutes... Sounds unbelievable .. I think it is ..

Google produces Innovation machines and IT vendors set up factories… So back to Factories and start oiling machines. Probably we should change IT designations to map roles on assembly line shop floor … - foreman, supervisor (this is there already in some sense), operator, inventory manager, assembler…. As few expect, IT work will eventually be a factory job. For one, my wife and kids would be happy. I can then work in shifts and come back home at sharp, say 5:30.

It is interesting to watch what Google is doing ….though their innovation factory recently had an outage…. With “Buzz” having “bugs” related to privacy...

You know what … as a tester, bugs make me happy… that is an eccentric , one sided view …especially with those who boast “machines and factories” of software. After all we are very far from the day when truly software will be produced in machines, requirements will be read from humans through bar-code type readers, we would be able recall software as cars (instead of sending a patch) and there would software garages.

Dreaming …!!! Someone woke me up... It is a brand new Monday….

Shrini

I’m blocked… but why?

"There is no such thing as writer's block for writers whose standards are low enough." American Poet William Stafford

I am blocked from doing any interesting writing for over 2 month or so. And this post is all about the condition of being blocked in writing.

As paradoxical as it sounds, I have not been able to write anything to my satisfaction so that I can publish. In the world of writing, they say it is a condition called "writer's block". It's a terrible feeling for me as block keeps haunting on every attempt to complete a half finished work or a new one. Here I am sitting with over a dozen of unfinished posts, like a kid confused about which toy to pick and play. It is funny that not long ago, I was chatting with Parimala Shankaraiah, a (or the?) curious tester about this condition called "writer's block". I suggested to her that when blocked, try writing about it. That is what I am exactly doing now. To illustrate the influence of WB on me, I am planning to keep this post small and focused on the topic and close it as soon as possible.

For non starters, WB is a temporary mental condition of extreme self censorship leading to rejection of semi finished parts of writing as this reference suggests. Keyword here is self imposed censorship. Those getting stuck in WB, can take conform in William Stafford's statement consoling themselves with pat on their backs for their high standards of writing.

Why I am blocked on writing?

While I am not short of ideas to write, when I pick up something to write after writing few paragraphs, I start getting a strange feeling that I am repeating or the plot of writing is hypnotically drawn into a predetermined destiny. That is where I stop writing. Not sure what is happening.

WB, like inattentional blindness appears to occur when you are ultra/super focused on a topic so that you literally stop thinking. I am not sure, my state of WB has anything to do my focus (or lack of it) on writing. As defocusing tactic, I switched to twitter and tried my hand there sufficiently enough to get an attention from none other than Jon Bach. That still leaves me high and dry and I am back to WB. Another thing that I suggested to self – stop writing but speak (do pod casts) or draw/paint (cartoons etc). I need to try these channels to see if they help. I think what happens in WB is one (of several) channels of expression of thought gets blocked while keeping others intact.

One thing is sure; I am under pressure of not being able to write even though I have many "good" ideas to write. Some say WB leads to lack of motivation or happens due to lack of motivation. I am pretty sure that is not the case with me. Anxiety, fear of being ignored or failure certainly is back of my mind.

Aiming for High standards? High degree of censorship? Yes, that appears to be the case. I reject lots of writing (pile of trashed pieces of writing) as they don't qualify my own criteria to be good enough to be posted.

I don't agree to generic prescriptions like Lack of preparation, reading, and picking up boring topic leads to WB. My experience of WB is that I am blocked on those topics I deeply interested in and dying to writing something significant, lack of preparation/reading might be an issue in some cases but the in case of the ideas that are pretty original, I often avoid reading to get my flow of my "unbiased" ideas.

Well, now I am suddenly realized that I am supposed to suffering from WB…let me abruptly end this post and start mulling over the cures for WB.

Oh!!! What a relief. This post comes to an end. Have I overcome writers block? Over to you…dear reader!!!

Note: I am surprised to note that there are other types of writer's blocks such as this and this. Here the word block is intended as "collection" or "store" of useful writer tools. Same name different connotation.

Why GUI Test Automation is Popular and Tempting? Part I - Business context

It is rare to someone on testing in IT world not be aware of "QTP, WinRunner, Rational Robot etc ". For the most in IT world, the word automation is synonymous with GUI automation. Perhaps some even might ask - "Are there any other forms of automation other GUI automation". Lots of how IT world sees automation has to do with how they see testing itself. You might have heard this "celebrated" statement - "Regression testing is more suitable to do automation (read GUI) so that we don't have to have our IT staff do that unnecessary testing manually".

To my surprise, having been associated with IT for many years now, people here still believe in perfect requirement, perfect test case and hence "doing it right first time". In contrast to this, software product world (likes of Microsoft, Google) treats test automation and hence testing in a total different way. These two probably operate from two extreme ends of automation spectrum. While discussion how automation is perceived and realized in IT and software product world can be a separate post in itself, I propose to elaborate here my views on the popularity on GUI Automation in IT.

Testing is necessary evil and should get done quickly

This is the reason #1 that drives GUI automation in IT. Business spends on getting IT application features developed and hence often fails to see a reason why they should spend on "testing" at all. Business feels that IT should get their acts together, get aligned to business (hence IT-business alignment is often a well understood but poorly executed theme) and use the money towards delivering business value. I even have heard some business folks saying "We pay for the application features not for their testing". Hence funding testing becomes IT's problem. That is why they look for solutions to reduce testing (or spend on it). As simple as it sounds, IT pays for testing (not the business)

Testers (read business users or analysts) are hard to find and they hate regression testing

Traditionally, in IT business analysts (aka BA's) are hot commodities. They do most of what we call as "testing". With outsourcing wave sweeping IT, some of BA roles are taken by either BA's from outsourcing vendors or some junior tester working under supervision of BA through highly scripted test procedures. The problem is that these BA resources are few and are in great demand. But as software releases happen, changes come in – testing (regression testing) has to happen and BA's hate regression testing (or all of testing). So, an IT manager needs to find a way out to carry out this regression testing somehow.

How to get testing done fast and cheaper?

If you want to do something fast, give it to machine or a computer program - goes a typical approach of an IT manager. So automation could be the answer. If you position automation as an aid to reduce the cycle time to market chances of you getting funding is very high. Surprisingly this claim is so powerful that when made often goes unchallenged. The magic word "reduction of cycle time" (that too without any qualification) is tempting for many that it will make them to completely overlook bottlenecks in development (programming), bug fixing, investigation and any other non-testing activities. Thus speaking in terms of business language is so important in proliferation and apparent success of GUI automation in IT.

With GUI automation, you can easily make a business case for automation.

To me, the terms like test cases, regression testing, test cycle time etc are the ones that business stakeholders appear to understand very well. As compared to unit tests and other forms of non GUI automation, you can make your case for investment in GUI automation to business stakeholders. Let us say you have 5000 test cases for a regression testing to cover that takes 5 person days to complete, automate 50% of them you can straight way knock off 50% of your testing cycle time reduced. This claim due to its simple math and logic makes a compelling case for GUI automation or simply "automation". What else can be more appealing than investing once in automation (for the sake of making the business case and further forever – down play the maintenance costs) and saving on a percentage proportional amount of automation achieved for 100's of cycles in the future?

Tool vendors help you by talking in business language

Enter tool vendors – with the fancy looking charts, scary quotes like "more than 60% of project cost goes to testing, what are you doing to reduce it" and likes Gartner's and Forrester's to support the claims with numbers, life for an IT manager in charge of testing has never been so easy. Just buy a tool, engage an outsourcing vendor to do GUI automation – all your testing worries are over in one go. Dreaming never ends here. Take a look at any brochures, websites, commercial literatures – they are full of business terms like ROI, Cost of testing, business impact, and Time to market and so on - making a perfect connect with those that matter in making decisions about spending.

Now try doing this (or these) with non GUI tests such as API tests or xUnit framework based unit tests – what will business say? What will IT managers say – "well, developers got to do that testing anyway"? So there is no apparent business case to make so sell automation. Writing unit tests will not speedup your testing cycle, will not reduce time to market, might reduce some portion of that "boring" regression testing – but. So you have no financially oriented incentive to do any automation that is non GUI. Since GUI automation paradigm as so much hardened into dogma – very few challenge it.

If you are the one working in a software product company – you would say "do less of GUI automation, they are very brittle and costly to maintain". You would be surprise to know/hear – the opposite works (or at least appear to work) in IT world.

It is "power of talking in business language, very few challengers for the claims made about automation and excellent support by tool vendors" that makes GUI automation popular and tempting to attempt. Even if you fail – you can blame it on almost anyone or anything around – test cases, tool, automation folks, application changes, changing requirements, test data, operating system patches and so on.

To be continued in Part II

Eurostar 2009 – A Trip Report

I am just back from EURO STAR at Stockholm. As a ritual, and like an obedient delegate and speaker, I am narrating the experience. It was my first visit to Sweden, home for Alfred Nobel. The conference started with a bang – welcome speech by Dot Graham, the program chair for 2009 edition of biggest European software testing event. After her being first program chair at 17 years ago, she still appear to breathe same energy of someone who is attending her first testing conference.

Lee Copeland, the veteran of testing conferences in the US, who followed Dot as a key note, was at his usual calm and composed best. He spoke on next few big things in testing- a good list to watch out for. His talk also contained slides on good books for the testers to read. Probably it has become a habit for him to promote his book on "Test design" in each of this talk … (I have heard 3-4 of them). He might call it as "Shameless plug" or "No… I didn't mean it include it here", I think he should stop doing it as the book sells on its own merit. Thereafter there were parallel tracks and everyone had a choice to go to the talk that they wanted. This time I like many others, had the power of twitter to spread information about conference, with the hash tag #esconfs, I was one of those many active twitters using tweeting about the talks, and happenings at the conference. Check out all tweets related to this conference here.

As it happens in every conference, much of the action was happening outside the track sessions, people meeting, exchanging cards, and fiercely debating on topics that they hold to their hearts. There I was, roaming around trying to see which group to join. Eurostar Test Lab was in interesting addition to this year's conference, run by James Lindsey and Bart Knaack - was a big distraction (constantly pulling passionate testers towards it and away from the track sessions). I spent few hours of testing and spent some good time with James Lindsey one of my favorite exploratory testing proponent. I logged few bugs too. It was a good experience. James reminded me to explore and find out when I had a question about a feature of the application that I am testing. Here is a report from Michael Bolton on the lab

Michael Bolton was an attraction, true to his image of great speaker and testing wizard. I rarely saw him alone, always surrounded by few people, Michael was just keeping them engaged with his teaser questions and people kept coming to him. His talk - "Burning issues of Day" was a presentation inspired by collection of white board statements of conference attendees, presented in an immaculate MB style. He took mock at several testing folklores and argued convincingly against standardize and over structured software testing practices. One thing that was really remarkable in that talk was Michael's ease and confidence - with which he entered the podium and started the presentation. He was at his total ease. Many wannabe speakers should watch him starting a speech and finishing one. To me, Michael gave an example of "what it means to be being oneself". Good learning there.

It was great to meet/see likes of Jonathan Kohl, James Lindsey, Fiona Charles and Ray Arell. In the conference made few new friends – Johan Johansson, Tobias Fors, Daniel and others. That evening we got together with and headed for a dinner and followed by a meet up at a friend Pabalo's house. That was a chance for me and others to have a go at musical instruments. I settled at Drums, Michael and Pabalo took guitars. The concert went until 12-1 Am. I decided to call it a day as I need to catch up with some sleep before my conference talk next day.

It was a big day for me. Getting to speak at Eurostar has been my dream. My talk was about metrics and how they can be dangerous if used inappropriately. I wasn't nervous and was sure that I will speak my heart. I requested Ray Arell and Fiona to help me with few photos. Michael's and few others presence at my talk, helped. I managed to finish my 15 odd slides at one dot – 39 minutes. Few questions and it ended with applause. I was greatly relived doing my duty at the conference. I plan to write a separate post about that later.

I met this guy Daniel in the conference– he did not carry his business card hence do not know his coordinates. Daniel and I argued for hours about the nature of the software – physical or abstraction. Daniel insisted that software has a 3D physical existence. He even said that he had confirmed this with 2-3 physicists he knew. Software when reduced to its "atoms" or basic units is some sequences of 1's and 0's that are stored on a magnetic material. When we power the computer or a computer like devices, electric current passes through transistors and other electronic components and bring the software to life. Daniel argued that hardware components that held the sequence were real and physical. We could never take the argument to the conclusion. But it was a good argument.

Third day of the conference started with keynote on agile adoption at Intel and experience sharing by Ray Arell. It was a good session with lots of advice on getting agile right. I had to do lots of running between office and the conference venue (yes, I had some official meetings set-up on that day). I track-chaired a session by Aslak Hellesoy on "Cucumber". The session was good and well received. The name cucumber look puzzling to me as why "Cucumber" – Aslak clarified during the presentation that it was the name suggested to him by his fiancé/girlfriend. It is become fashion to name the tools/frameworks companies by totally unrelated names. Steve jobs success with Apple continues to inspire many to name their creations with the names that you typically would never make a connection. In the evening there usual conference rituals like panel discussions, prizes, vote of thanks and announcement of next year's track chair – John Fodeh. Zeger van Hese's paper wins the best paper award, Naomi Karten wins the best tutorial award and Michael Bolton wins best bug of conference. Michael wrote about it here.

In all, it was an excellent experience of being at Eurostar 2009 and at Stockholm. I will carry many memories from it for years to come… Thanks Eurostar!!!!

Don't forget to check out the photo gallery

Check out few reports about this conference

Female Funtestic Fanatic

Rikard Edgren's Test Eye

Star Tester issue 45

Note: Narrating a story or an event like a conference is tough, I think. I attempted to do it see if you like my story.

 

Defining the problem : How business works

Lee Jack, a software testing consultant/IT services Sales Manager is tensed about his proposed meeting with Mark Johnson, a Senior IT manager for a large IT group. Lee, after some positive signals in a cold call, is looking forward to sell testing services to Mark's group and hence to his entire IT group. Here is how the conversation goes.

Lee Jack: What is your problem statement?
Mark Johnson: Not sure. In our recent conversation – you mentioned that your company provides Testing services - what you can offer me?


LJ: You need to focus on Independent testing
MJ: We have been managing with developers doing testing for last 6 years – have no problems whatsoever. Why do I need to bother now?

LJ: May be your developer testing is not proper or inadequate or it is costly?
MJ: How can you say that?

LJ: Tell us about production defects and quality of your applications in general…
MJ: I told you already. We have no production problems that we worry about. Our site is every now and then goes down. We wait for 30-40 mins and start over again everything works fine. Why bother?

LJ: Do you say your applications are of high quality?
MJ: I would say our applications are of “good enough quality”. I get what I pay for and I have the quality for the application that my users care for. So far so good.

LJ: What are your “business plans” going forward? New growth etc? What you seem to be saying “good enough may not be good in the future?
MJ: May be. I don’t think that far ….

MJ; Have you thought about cost of testing? Do you have any mandate from your boss about reducing IT budget and hence reduced budget for testing?
MJ: Oh!! Yes. We have taken care of the larger part of cost problems. We have nearly replaced all our commercial software stack by open source equivalents. Same is true even for tools. So we can say we run our entire shop on open source tools. That is our achievement.

LJ: Tell us about your development /testing model ?
MJ: We are pretty advanced from that angle. We follow scrum based agile development model – outsourced. Developers do all required testing and development and everything is fine. I am hearing you talking about QA again and again without me asking about it. We do agile model and as such there is no distinction between development and QA. As such we do not think cost is our concern.


LJ: I did not say “QA”, I said “testing” – I suppose you know the difference.
MJ: Fine … here we use these terms interchangeably. Well, does that matter in agile?

LJ: How can we help you?
MJ: You have been saying that you give QA , sorry testing services – Do you provide white box testing services?

LJ: Our Testers are trained well in both black box and white box techniques – that should not be a problem.
MJ: Let me make it very clear – in the name of QA or Testing, I don’t want some so called tester coming and doing “negative” testing and showing 100’s bogus UI level bugs. That is not what I want.
LJ: Our testers are trained in exploratory testing and can flush out hidden bugs
MJ: I said I do not want negative testing – UI level keyboard banging. I want white box testers.

LJ: What according to you is white box testing?
MJ: I am surprised that you are asking this silly question. It is the testing done with the knowledge of product internals. Something that developers do. Don’t you have such testers .. I mean developers?

LJ: Well, the reason I asked this question is – there are no universally accepted meanings of the term white box testing and I wanted to make sure I understand your version. OK Why can’t your current developers do that?
MJ: Our developers are already stressed out. We have very well defined agile processes. Our developers do very smart and JUnit based automated tests are our key to success.

LJ: OK … What can we do for you? It seems that you have everything that you need.
MJ: How about your folks doing an assessment and tell us what our maturity is? How well we are placed with respect to industry standards?

LJ: That appears like a different problem …. Anyway, are you sure You want an assessment?
MJ: Yes. I would like our processes to be best as per industry standards.

LJ: What are your expectations out of this assessment?
MJ: We need to add few white box testers to our existing team as we are weak in that area. We are sure we do not need QA resources that do negative or exploratory testers. We are also sure we do not need testers. We need developers who can do white box testing.

LJ: Oh!!!! That is what you want? I can give you developers who can do white box testing? Why you need assessment?
MJ: You see, we are looking for a consultant to make case for us to get budget for these resources. We really do not worry about testing, QA, White/black box testing etc. Can you get a consultant?

LJ: Great … I will have a top notch consultant at your office tomorrow.
MJ: Thank you. Make sure, you get someone who understands our problem very well. I don’t want to go through all these questions again.

LJ: So, let me recap. What is your problem?
MJ: Damn it …. I need a business case for increasing my team’s headcount and I need white box testers. Are you clear?

LJ: Yes … Sure …
MJ: Thank you very much.

What do you think … what is the problem here?

Update: I did a mistake of swapping the names of MJ and LJ. This is corrected. See how it reads now.

Shrini

Necessary Tester skills ....

Curiosity? Keen and close observation? Skepticism? Creativity? Arguing/debating? Imaginative? Analytical and Holistic? Good at Probing/Investigation?

None – would say many IT testers, I meet on regular basis

Today’s IT testers have forgotten these terms I suppose. When looked at macroscopic and group level – todays Test managers, delivery heads appear to have lost sight on these core skills for testers. When I ask any typical IT testers what they are doing to enhance the skills, typical answers I get are - learning domain, usage of specific tool, or a programming language, or some process/methodologies stuff. That is not wrong but having any of these skills without the basic skills required for testers – we would be developing the armies of robots who are very good at following orders and over a period of time, stop thinking on their own.

NY Police (a profession similar to testers) are taking course in “observing and describing” – What our testers should be doing to sharpen their testing skills? How they are putting their cognitive skills in their day today work?

http://www.smithsonianmag.com/arts-culture/Teaching-Cops-to-See.html

Shrini

A process is what you actually do. A process document describes what someone would like you to do, ideally. They rarely coincide exactly, and sometimes don't overlap at all - Jerry Weinberg

Who decides what is a bug and what should be fixed?

I was submitting a proposal for a paper for STC 2009 conference. After completing all fields – about 20 of them , I accidently hit clear button (by practice – “OK” or “submit” button appears first in such forms) and all that I entered is “gone” – worst there is no way to recover.


Is this a bug?

Another catch – for the date field – no format is specified, neither there is a calendar control. How do I find out the required format? Try one and get to know about the format.

Is this a bug?

If you were a tester who would strictly go by “test cases” generated out of “specifications” – you are likely to miss such bugs – remember there is something called “Requirement based testing”. Alternatively, you might argue that these are not bugs or bugs of low severity. Who has the final authority to say which is a bug and which one should be fixed and when?

Shrini

Your user acceptance testing is fixed – Is that a problem?

“This troublesome user acceptance testing got to be easier. It is really a pain” said of my colleague quoting an IT manager responsible for release management and user acceptance testing. Major problem according the IT manager was getting the “users” to do their job – testing a defect fix, feature enhancement or a new release. Users are always busy with their “day job” and for most of them doing testing is a low priority activity. I have heard many similar stories regarding this important aspect of “end user” responsibility.

In traditional IT organizations, user acceptance testing is a formal stage in the testing life cycle where business users will test the proposed changes to production software applications. IT organization that delivers the software to meet the needs of business users, require a (formal) approval of proposed changes that come in terms of defect fixes, enhancements and new releases. Since it is business users that ask for changes for their existing and fund the development/testing work – they will have the final say on “acceptance” of proposed changes to production applications.

Primary purpose of user acceptance testing is check (cursory- final) proposed changes to the production software by those users that asked for the changes so that any last minutes changes and surprises are avoided. User acceptance test is the last gate before software goes Live so that it provides last opportunity for all involved (both IT and business) to make corrections if required. Depending upon the nature of business supported, nature of software application, nature of changes proposed UAT may last for few days to few weeks. Software that fails in UAT is typically assumed to be insufficiently tested and is variably returned to IT for fixes and more testing.

The problem of UAT
Business users who hold the responsibility of approving software updates to production systems, typically are not testers and as such do not come with testing skills. Depending upon the nature of software updates, IT team will ask business users to carry out testing to check the features of new system. Most of the user acceptance testing tends to be repetitive and that is what drives “real users” crazy. For development team (IT), their work is not complete until the piece of code is user tested and accepted. Hence they literally chase user group to perform UAT with users complaining about the time crunch and their “important” business work getting impacted. This creates a situation where both parties (IT and business) want UAT to be somehow completed so that each can carry out their business as usual. Hence UAT often gets “fixed” with the consent of both parties having stakes in the activity.

Forms of fixing UAT
Between Business and IT, UAT can be fixed in number of ways – some of these are reasonably business driven models given the constraints.
1. UAT will be performed by the members from IT team where business only reviews the results of the test. If the results are OK then UAT is ought to have been completed
2. UAT will be performed by a third party, such as someone from support staff. Business will review the results and accept the proposed changes if results are OK
3. Business will provide a prescribed set of test scripts that can either be used by anyone IT staff or support staff. Results will be verified by the Business.
4. UAT will be done like a demo to the users where IT staff will execute some pre-approved test scenarios related to proposed changes.
5. IT team will train the business in new features proposed to be introduced. Business users after training, test (use) the proposed software and accept the software

Why fixing user acceptance testing is bad thing?

Why bother to UAT after all? For IT, more often than not, it is a formality to be completed before they push the code to production.

In my opinion, it is the spirit and purpose of UAT that gets compromised. Typically, in spite of all best efforts, the depth and frequency of interactions between IT and business throughout the project remains low. When business users do not participate with full spirit in UAT, lots of things go unnoticed into production. This might results users (non participating ones especially) getting surprised when they see the product.

What has been your experience? Do you bother if you see your UAT is fixed?

Shrini

Tom DeMarco’s confession

Confession is probably a harsh to describe what Tom DeMarco, the creator of celebrated punch line of software managers – "You can't control that you can't measure", wrote recently. But, if you read Tom's recent article "Software Engineering – An idea whose time has come and Gone?" in IEEE software, you would probably say something similar to this. In this short 2 Page article, you will find Tom DeMarco in reflective and retrospective mood.

In the book "Controlling Software Projects: Management, Measurement, and Estimation (Prentice Hall/Yourdon Press, 1982) ", Tom talked about "Controlling" and "measuring" in Software Engineering. Nearly 40 years later, he now appears to admit that he pushed the notion of "control" and "measurement" too much.

This "confession" has apparently caught the attention of many. Jeff Atwood writes an obituary to "software engineering". More than his post, the comments for the post of Jeff Atwood are interesting to read. How come, suddenly so many are accepting now that "software" is human centric, people oriented, "engineering" is not the right term to use and so on? Michael Bolton's recent article (three kinds of measurements and two ways of using them) on stickyminds appears have been triggered by Tom's "confession". Matt Heusser writes about metrics here, here and here.

Managing vs controlling

In his own admission, Tom seems to distinguish between controlling and managing. He gives the example of "upbringing of a teenager". As any family therapist would recommend, you manage your teenage kid rather than controlling them. Like in most human endeavors (including software programming and testing), you can manage quite lot things than controlling them. I think we can manage (and also a control a bit) WITHOUT measuring ANYTHING at all. I like Matt Heusser's example of hair cut. Many of us control and manage our hair (style) without measuring.

Manage people and Control Money and timelines

This is Tom's recipe for managing the project without controlling it. While breaking project into human elements and non human elements (such as code, schedule, money) is a welcome change, I am not sure how can you do it – managing people and control money and timelines. To me, these two sets of things are NOT mutually exclusive so that you can treat each in a different way. Controlling time and money impacts people and managing people impacts timelines and money.

Some software is really engineered!!!

Dave Markle makes an interesting point in Jeff Atwood's post - "IMO you can't say that programming languages themselves aren't engineered based on solid computer science. You can't say that something like LINQ hasn't been engineered. Whenever you use a FSM in your software, you are applying computer science, which makes you a software engineer"

Programming languages are engineered, operating systems are engineered and so are software algorithms. It probably the user mass/ size of the group decides engineering vs. crafted.

Dave uses the Paint analogy nicely to drive home the point - the paint artist uses is engineered (developed and mass produced using the principles of chemistry and physics) where as the "art" produced by the artist is NOT. This stirs the nest of debate of what is engineering and what is craftsmanship. Are Engineering and Craftsmanship are mutually exclusive? I am afraid NOT.

Tom, Damage has been done and is still happening. Many still many abuse your punch-line to push loads of documentation, process, approvals, and meetings and of course endless charts/graphs of metrics – all in the name of "control". Probably the time has come to step back and be sensible on "measurements" in software.

Shrini

To tweet or to blog ...?

Due to me crazy traveling and work, I have not been able to write as frequently as I would have liked, for the blog. At least half a dozen potential blog posts are waiting to see the light of the day from my side. It is just not working out.

I have been on twitter (for starters - you can take this as a quick blogging or microblogging) quite active in recent days. Happy to see many following me now. It is suiting me for now as I need not feel guilty of not being able to discharge my duties as a blogzen (no!!! this word has not yet been used by someone previously)of software testing blogosphere.

So till, I get to full time blogging - please follow my thoughts on twitter. I have added twitter feed on this blog to facilitate for my readers to catch up with what I am working on ....

Thanks for being my blog reader ....

Shrini

10 ways to make automation difficult or ineffective

Here goes another 10 items list for automation. If you are in IT or IT services space and manage/deliver automation solutions -- make sure you stay away from these as these items have high likely hood of making automation inefficient/ineffective and difficult.

This list is an extension of a topic and this list (of 10 items again) for test automation outsourcing

10. Wild Desire to automate 100%

9.Attempting to automate existing test cases without scrutinizing them for “suitability” to automate

8. Mapping test case to script 1:1 linear model – falling prey to deceptive traceability and gold plated reporting.

7.Not building automation solution bottom-up , unidentifiable building block of the solution.

6. Trying only one type of automation or attacking only one layer of the application – Farther you go from code, messier it gets.

5. Focusing only test execution related tasks

4. Treating automation as scripting – ignoring “generally accepted good software development practices for hygiene.

3. Failure to involve developers from the beginning – Not attempting to testability or automatability of the application.

2. Jumping to automation to speed up testing or save cost before fixing testing problems – inadequate, inefficient and broken.

1. Failure to arrive (formulate) at the right mix of human testing and automated test execution.

0. Using Automation as solution to testing problems.

I reiterate that these are applicable mostly to COTS driven, GUI functional Testing automation that is typical in IT/IT services environments. WI might have to rewrite some of these for xUnit type formalized unit testing (that is also automation and some call it even as "testing").

Shrini

Is this a bug?

I was flying from Cape Town to Bangalore through emirates flight. It is convenient to online check-in. I do it as I can choose an aisle seat. But for the second time, I got into problem while doing online checking for emirates. Probably the Internet connection was slow – in both occasions, emirates online application did not respond and I had to close the browser after 5-10 minutes of frustrating wait – staring at screen.

Here is the bug that frustrated me…

· I try to do an online check in and would like to change the seats.
· Application hangs when trying to save the changes.
· Close the browser.
· Try again to do check-in
· Get a message that the passengers have been checked in.

Fine – how will I know what are my seat numbers? How do I view my check-in details.

Apparently there are no easily reachable ways to gather information. Probably there is none. How do I search where is “view check-in details” or “view eBoarding Pass” or something similar? I tried site map, tried “Help” and tried “search”… could not figure out the link for viewing check-in details.


Is this a bug? If you are a tester will you catch this bug? If you are a developer will you accept that this is a bug? I am sure most people will say “if this is an intended functionality (I think, it is), then it should be documented requirement specifications. Once it is there, tester can write the test case and developer will make sure that the functionality is coded and tested”. Some testers might say this is “nice to have feature” …

What might have happened here? Requirements problem? Development problem? Or a Testing problem?

10 ways to reduce cost of software testing

In current economic situations, IT folks worry about one thing – “reduce cost”. I have been frequently asked “how to reduce testing cost”. A no brainer answer would “do not do testing … at all”. How many buy this idea … can current breed of IT applications sustain with less or no testing at all? When I use the term testing –I am referring to “non-programmer” testing.

Here is my draft list of suggestions ...

1. Closely work with developers, do some parallel testing with them as the product/feature is getting developed
2. Identify and eliminate non-testing activities that occur in the name of process, documentation, management, metrics etc.
3. Analyze and profile every application under the portfolio to determine “stable” and “well tested” areas of the application. These areas should receive the least or no testing effort.
4. Analyze the test scripts suite and remove redundant, worn out ones. Aim to reduce scripted test repository as small as you can.
5. Review and reduce “regression testing” on the basis of “well tested/stable areas” of the application
6. Switch from resource intensive and highly scripted testing approach to highly improvisational exploratory /rapid testing approaches
7. Plan testing in small but frequent cycles (Session based exploratory testing approach) – reduce planning and management overheads
8. Analyze and reduce the usage of costly tool licenses - especially those do not help in testing directly (test management tools)
9. Cut down on lengthy test plans, testing reports, dashboards – switch to simple but frequent test reporting.
10. Simplify defect management process – reduce defect life cycle – resort to informal/quick defect communication.


Some this advice might look like a simple common sense (eliminate waste, focus on tasks that impact end result DIRECTLY). With so much selling happening about “testing tools”, “factory models”, “cheap and best testing services” – any common sense is difficult to come by.

How would IT community react to these suggestions – most likely response would “This would not work, how can we reduce testing, those test cases, processes, metrics, management practice?”. These suggestions would be most likely to be rejected on the grounds that testing cost needs to be reduced without “compromising quality”. Many IT folks think that quality comes from test scripts, processes, metrics, testing tools, automation etc. I am afraid quality is not such a simple thing.
Again, there are no free lunches here … if you are thinking about reducing cost of testing, there are always risks of impacting quality (roughly goodness or confidence in the product) in one or other way. If you approach the problem (cost vs quality) from a quality side (improve testing - test better, deeper and wider), then the chances of achieving good quality and also “some” cost benefits are more likely. However, if you approach it from “cost” side of the equation – you might do achieve that albeit some impact on overall goodness/quality of delivered product.

Note that some suggestions mentioned in this list call for some smart testers who can think on their feet, work with least supervision, least (optimum) documentation and processes and so on. I think, the focus should shift from process, tools, management, documentation to Skill. There can be problems in getting such resources in IT scenario (especially in outsourced/offshored world)

You have choice … which side you would like to approach the problem ..?

[update 20/Apr/2009] A colleague of mine reacted to this list saying "These are too risky suggestions and he would not recommend any of these. Business prudence is totally missing".

I think he was expecting to see some "low risk and high return" type of suggestions - like those "cheap" and "best" items. I still do not understand - there can be no risk free ways of reducing (testing) cost -unless you are totally spending like crazy without any thinking. We do not seem to have such risk free - free lunches - why fool ourselves and the client in believing such "non existent" things?

Another suggestion that came up was "Let us use standardized processes". How standardization can reduce cost? what is the cost bringing in standardization itself?
May be the expectation is that standardization will make each tester behave identical to another like robots. Are robots cheaper? may be? may be not .... They at least do not whine about working on week ends :)

Shrini