I am a sapient Tester …

Here is my first and a very crude attempt write a poem (if it can be called as a poem) on sapient tester ...

I understand that testing is a sapient process *
(* A sapient process is any process that relies on skilled humans)
I question things around me
I model things to understand their behavior
I learn from related fields in testing (ones that are close require higher levels of human cognition)
I use my brain and I constantly try to improve my thinking
I don’t believe in rote processes that mechanize human thinking
I don’t use documentation, KA/KT* as essentials for getting started with testing
I can figure out my way in understanding the software I am testing.
I use automation as aid in manual testing not as a replacement
I use techniques as heuristics
I just don’t simply follow the written instructions and procedures - I question them
I collaborate with developers, other testers, users, support personnel and other stakeholders to understand wide range of perspectives.
I understand depth and value of human testing
I understand that by automating a test, I might be loosing some thing.
I am sapient tester... Are you?

Association of software testing (AST) publishes a magazine called "sapient testing"

James Bach wrote about sapient processes here

* KA - knowledge Acquisition and KT - knowledge Transfer

Shrini

Thinking about Zero defect software - Attaining software Nirvana ...

“Humans have always proved impossible is possible … Dreaming, Desire and passion is key to achieve “impossible”. Wright Brothers dreamt about “flying” – they achieved it. Once, reaching moon was impossible – now? You need to give your life to it …Zero defect software is possible …” You need to aim high”.

This is how, a colleague of mine argued with me when we were discussing about Zero defect software. He believed that many software like IBM mainframe apps are running with near zero defects … (there could be defects but of cosmetic nature)

The notion of defects, number…

No defect is bigger or smaller … Let us say there is a 0.5 second delay in system response … Will this matter? Is this a bug … depends upon which software we are talking about? Context is important …

When people talk about software processes, discipline and making an attempt to achieve zero defect … they are often of the opinion that …
Human beings make mistakes deliberately … many or all can be avoided if there is a second eye or a watch dog
Human do sloppy work unless controlled – given a choice no one would do a good job (if no one is observing)
Humans require constrained and regulated environment
Humans - overlook

It is this possibility that people require discipline makes others (mangers especially) to think that zero defect is possible. People who vociferously argue in favor of zero defect software – think so because they feel that it is because of human laziness and other related aspects – defects are introduced. Put a system of governance, policing – you will achieve zero defect. How true is this?

Chasing the definitions of Quality ....

Why do we have so many meanings/interpretations/definitions of quality - A colleague of mine asked recently. I did not have any ready answer for him ... I said ... after a pause ... "There are so many ways to see the quality". Because, I think every one, whoever attempted to define quality (from Juran to Weinberg, ISO to CMMI), did so wearing a "user's" hat while they were not real users. It is like doing a role play temporarily - a third party view.

Michael Bolton mentioned this elegantly...

Quality is not an attribute of a thing or a thing (most of us believe that is in a thing). It is a relationship between the user and the product.

So extending Michael’s view point - quality is a two way and intimate relationship between the user and product. There is no room for third party.

So as a third party everyone can come up with "possible" meanings of the term quality.

The moment we understand that it is relationship on a timescale (perception of quality does change over time) not something built into the thing ... we will be good. Beware a notion of good quality does not alway remain so .... as time changes so does the perception about quality in the minds of the beholder.

What does this mean to we, testers?

- As testers we do a third party assessment of (to best of our abilities) the relation ship between a product and it's anticipated user base.
- Things like fast, great UI, cheap, reliable are only manifestations of relationship between user and product - not real things or attributes about quality.

- Quality is observer dependent - quality can be viewed always and only viewed from the eyes of the user's frame of reference
- User's taste, perception of quality for a product changes. Hence there can not be any "one-time", "fits all size" kind of testing/assessment for a product (more so .. if it a software product)

Who is next in the line with a definition of quality?

Shrini

When method dictates outcome and Goal – See Goal displacement in action - I

I first heard (came to know) about this word from James Bach and this immediately caught my attention. I started seeing it everywhere. The phrase "goal displacement" is generally referred to a situation where method of achieving something dictates the out come and goal. The process by which the means used to achieve a goal becomes more important than the goal itself.

Here is an example that I think illustrates what I mean by “Goal Displacement”.

A model of Test Automation

Manager: I am quite impressed with overall progress this automation project. But I would like see some metrics and measurements to quantitatively monitor future projects in this area – like number of scripts produced per day, individual automation engineer’s productivity etc
Me: Ummm … right now, our model of automation development does not itself to such measurements. We don’t treat overall work in terms of scripts and measure engineer’s productivity. I am afraid; I don’t have an easy answer to your question

Manager: I understand your concerns. But I need those metrics.
Me: Here is a deal, how about modifying the whole process of automation such that it is easier to measure and exactly built according to metrics that we would like to measure (regardless of what outcome we would like produce)?

Manager: I did not understand that … Can you not just introduce them to existing model without affecting anything else - especially the output?
Me: To accommodate the kinds of things that you are asking, I will have to modify the way we develop things – which I am afraid will affect the quality of our deliverables. I don’t see any way to make those measurements in current model.

Manager: Metrics and measurements are not suppose to change the process instead I heard that metrics help to control and improve the process.
Me: In this case they do. Do you want me go ahead and change? I see a serious, negative side effect of changing the process/model to suite metrics.

Manager: Ummmm … Let us talk about it some other day (walked away)

What is happening here? Manager wanted to see some metrics to see how the automation team is performing. I simply said "You can have those metrics but at the cost of changing a process that is apparently working fine and producing the results that make you happy". So here, the introduction of metrics (or a measurement process) causes a shift in the goal of producing good automation code to goal of having a process that measurable in some way. It is like saying if current process (way of doing things) does not cater to a new related expectation, possibly a conflicting one with original goal (a displaced goal) - then forget about goal - do whatever change required to meet new goal or expectation.

Stay tuned for two more examples of goal displacement ...
Shrini

Software Testing certification - To be or not to be ..

I am yet open up my thoughts on certification ... also I have not been very critical of certification programs. Here is an attempt to publish my personal views on this topic ..

In 2003 I took CSTE exam ( A link from QAI is here) - as asked my manager and cleared with flying colors - 90% above ...My preparation for the exam was painful ... I had scratched the CBOK (reference material) all over as I did not agree with most of material - which seemed to copy paste from a software engg text book and about 20 years old stuff. I could memorize lots of stuff without sufficiently challenging it. I passed the exam ... did I learn anything new that helped my work --- NONE other than some terms and their meanings (questionable).

I also constantly watched the kinds of people who passed such exams .... I must the quality was bad. Passing was easy ....In my opinion, most of certifications (I am a certified Java programmer too) suffer from this problem. There is so much focus on memorizing things not about learning.... There is little space for debates and questioning... That is you questions do not have "comments" field .. or "I don't agree with premise of the question because...." kind of open ended response from the candidate .. Who has the time to site and evaluate all responses ...?

Discussions on certifications often take "Emotional" angle - people say exams like BE and certifications help one to improve skill and knowledge .. I would say they only provide reading material .. rest is in your hand (rather in mind) . I strongly feel that skill comes first and gets developed by practicing, doing, spending real good time with dedication not by passing an exam that tests your memory retention capabilities.

The problem I see with testing certifications is that - they don't test testers skill to do testing ... (a practical exam like our science lab in schools might help) instead they test tester's memory retention skills.

Are there any certification exams that watch tester testing stuff (with some video etc) and then give "good to go" Tag? No .. that would be expensive and tough exam to administer right ...

An example of goal displacement ... certification exams appear to be designed for easier and large scale (yes/No or choice) kind of assessment than complex/tougher assessment of watching tester in action and rating his/her level of skill.

Why goal displacement -- Some group of people in testing thought - we need an exam to test our testers .. so what kind of exam should we have? ... something that is easier to administer or something that tests tester in action... They prefer to choose the first one ... So the goal of having a good certification got shifted to having a certification exam that is easier to evaluate ...why ? Good certification exam requires complex models and involve lengthy/costly administration. It costs lots of money...

Having said all this ... its your call .. I am neither in favor or against to certifications in testing (some people 's daily bread comes by running those exams -- let us not take away that ...). It is a big business in testing industry today. Being testers, let us evaluate what is good for each one of us (considering each ones technical and educational background) and take the decision.

Any alternative to certification ....?
Focus on skill ... how many of us can go to an interview and challenge the interviewer "Give me a software to test ... give me an hour ... come back I will have a defect/issue list ready". How many of us can take this challenge of "testing anything, anywhere, under any time frame" ? Managers in IT companies need to see evidence that this person can test ... he has tester attitude...

In absence of such evidence, managers resort to something that easy to check (do you have this certification?). If you are fresher or beginner in software testing field ... and struggling to get a break or good job ... You have two options ... Get certified and get a job .. (there after it is your skill that keeps you on job not the certification) other option ... practice testing, read stuff, read books, learn programming language, debate, sharpen your analytical skills, write blog, discuss with others in the field .. build a credible profile for your self (Google should be able to find you) and go confidently to any interview and say "give me stuff to test ..."

First one is easy path ... some money and about few weeks of reading, take the exam your are done... second one is tough one.. Requires you to commit to the profession ... requires your learn things, may take years to be good at testing, many years to build a credible profile that speaks for itself ...

People choose first one mostly as it is easy ... So I would not say all who support certification are not skilled people and take short cut to success neither all those who oppose certification are good skilled testers …

You want to be certified? it is your call -- What ever you do, be a tester by heart - test everything/question everything until you are convinced

Read James Bach on Certification here

Michael Bolton on why I am not certified

Ben Simo on his favorite certification

Balancing on the other side -- AST certification debate

Shrini