Rex Black does not like James Bach’s definition of testing “Questioning a product in order to evaluate it”. I am not sure why. During test2008 conference, this issue was brought up during a discussion. Rex said (paraphrase) “Questioning a lifeless thing like software is bizarre. I cannot question my dog”. I attempted to catch-up with him later next day to see if I can know more about his views. When I managed to get his few minutes, he took me to a pillar (painted with red) and pointing his finger to the pillar, said “Are you red”? He continued “I am asking the pillar. Am I getting an answer? “Questioning software is ridiculous and bizarre”. I thought I would get a chance to react to what he said. Rex being a busy man and did not have time for “bizarre/meaningless” debates, excused himself and went away.
Is questioning software really bizarre? I don’t think so. When Rex walked up to that red pillar and asked “Are you red” – what was he doing? – questioning the pillar. Right? He did question the pillar. Through his eyes, he could figure out that it was a red pillar. What is the problem then? May be, he was referring to the inability of the “lifeless” pillar not to answer him back in some human language that he could understand. Well, that was answering part- not the questioning part. Let us apply this to software, everything that we do as a part of testing can be thought of as a question that we ask (not necessarily in the same way as humans communicate) and Software does answer (unlike the pillar) in a subtle way. Thinking about testing a questioning process is a strong and powerful way to organize the thought process about testing. As testers we must develop skills to question, skills to interpret the answers, skills to improvise questioning and skills to analyze the subtle answers given by the software.
What do you say Mr. Black?
[Update] More on questioning, meanings and various interpretations of "questioning" software by Michael Bolton is here.
Shrini
20 comments:
I hope some day have a 1 on 1 with Rex, I think they pretend that testing is something mechanical.
For sure the Pilar would not answer back with words, but it says a lot of what you see, even if you ask a little boy you had to understand his words and figure out what he try to say.
Sometimes people don't get used to think beyond the box or even inside the box.
I hope to get my chance to ask him about certifications and "functional" testing
Shrini - I think you'd be better off in life, and in testing, if you were to take Bach and Kaner off of the pedestals you have put them on and formed your own thoughts on these subjects.
The Oxford English Dictionary lists several meanings for the verb "question".
I'll give you this, he's being consistent: Rex is not even questioning his own chosen meaning of the word! :)
Another counterexample for Rex: The phrase "Question Authority", found on popular bumper stickers some time ago. Does Rex really think that expression means "Go find an authority and ask that person questions"?
I wonder if he's just being silly and deliberately dense.
Dear Anonymous ... thanks for advice to take of Bach and Kaner off of the pedestal. Let me explain - why keep them in my theory/practice of testing.
Over the years, I have developed a philosophy/ideology of testing. I must say larger part of it has been influenced by Bach and Kaner. Those of you who have interacted with me and know me by my writings, I do not accept anything just because someone says so. Before accepting anything into my testing philosophy, it (the idea) goes through a scrutiny. I believe, in my last 3-4 years of existence in blog-o-sphere, I have demonstrated this and have been consistent. Incidentally, Bach and Kaner's ideas on testing appear to be inline with my thinking. So I follow them by and large.
There is another reason, Bach and Kaner have this unique characteristic - they appreciate and promote "debates", "argumentation" and "questioning" to core. No doubts about it. That is true characteristic of a tester. I see very few in the testing world online (so that we can engage in communication with them) who promote such thinking. If there are any those I do not know of ... I would be happy to study and put their thoughts and ideas through my scrutiny. Kaner and Bach have not forced any one (to best of my knowledge) to follow them. In fact I have struggled for years to get into their company and get their attention. It is tough to get into their camp. I sense that those who feel threatened by the "aggression" that the Bach and Kaner (Bach especially) to the craft of testing and their commitment/service to testing community.
I have tried my best in all these years to write and share thoughts that are unique to my thinking and are original. While my thoughts on testing have been greatly influenced by Bach and Kaner - many of the ideas presented in my blog are my ORIGINAL ideas.
Let me ask you ... why you want me to junk Bach and Kaner's ideas and philosophy? Give me a credible reason - I might consider that.
Shrini
The whole concept of software is "ridiculous and bizarre". In the history of humanity, has there ever been such an intangible ( and pervasive ) tool used to solve the variety of tangible problems we encounter daily?
Our is the job of checking the magnetic math problems that virtualizes the human experience. So determining the veracity of these system requires a little departure from the traditionally sane and normal.
In my opinion, testing is not solely questioning the thing. It includes questioning our _understanding_ of the thing. If only determining quality was always as easy as figuring out the color of the office fixtures.
Oh.. Rex is President of ISTQB and chief promoter of its certifications..!! Now I see why this Bach bashing.. :)
Guru
Did Rex find it bizarre that you questioned his statement?
---Michael B.
>>> Did Rex find it bizarre that you questioned his statement?
I think he did Michael. As I mentioned, Rex appears to be a busy man and did not have time to engage in a debate with me. Apparently he had lots of other important things to do. Probably, he did not see a value in listening to my view point. He wanted to make a point to challenge Bach's definition which he did. I wanted to put a counter argument - I am putting that here in offline mode.
Shrini
After reading Shrini’s post, I wondered what ISTQB definition of Testing was? Well, below is their definition of testing. (Source: http://www.istqb.org/downloads/glossary-current.pdf pg -38)
Testing: The process consisting of all life cycle activities, both static and dynamic, concerned with planning, preparation and evaluation of software products and related work products to determine that they satisfy specified requirements, to demonstrate that they are fit for purpose and to detect defects.
- Could some one please explain what life cycle activities this definition is referring to? Isn’t the word life cycle related to living organisms?
- What does static/dynamic life activities mean?
- What are related work products?
- How do you satisfy specified requirements? Does requirement document tell you when it is satisfied?
- I am sorry (English is not my first language) I seriously could not understand this line “to demonstrate that they are fit for purpose and to detect defects.”
I feel Shrini has clarified the definition we acknowledge by. In fact, most of whom I am inspired (check my blog list from http://testtotester.blogspot.com/ ) are free enough to mail me back and help me, when I have a genuine problem or guide me or help me learn, if I prove my research & argument to them. So, now I request all ISTQBians to help me understand your definition.
I have also created a discussion thread at Test Republic on the same topic http://www.testrepublic.com/forum/topic/show?id=1178155%3ATopic%3A33849 . ISTQBians feel free to answer the definition is the forum as well.
-Sharath B
Sharath, I think in general, certified testers are taught to strictly adhere to following 2 rules.
Rule #1: Do not (dare) to question the authority (certification) body about definitions (especially). (Going by what Rex said - certification body is a non-living thing hence cannot be questioned)
Rule #2: Always remember rule #1.
This might be the case because, questioning definitions, meanings - disturbs the common nomenclature and vocabulary that certification bodies have defined and nurtured with so much effort and pain.
A classic case of goal displacement - having a common nomenclature and language is much more important than diversified set of definitions that are open for questioning and scrutiny.
Shrini
Hi Shrini,
Good Observation!
I feel what Rex meant was that instead of "questioning" the "software", we should question our thoughts/strategy/testing plan...
Maybe, its time we understood the phrase "Questioning Software"
Are we actually questioning the software? or are we questioning our thoughts?
I can't speak for Rex, but it sounds like he's just questioning your word choices - as you've said, you don't question software, what you are questioning are the *decisions* that went into the design and implementation of the software.
I think that Rex, like many other testers, is merely questioning your choice of words in the definition. I don't really see why this is such a huge deal - in fact, is it that different than questioning the term "best practices" (when there are no best practices - just good practices for a given context)?
Alan: no, I don't question the decisions. Not at first. I can't know what those are if all I am doing is testing the software.
I investigate what I see, what I think I see, and what I and others expect from the product. Because my knowledge is never perfect, I try to examine what I am expecting as well as what I think I am seeing.
It might turn out that when I raise issues there will be explanations from developers about decisions. But I'm not examining decisions while I'm testing the software. I might be making mental notes about what I hypothesize some decisions might have been.
Now, on the contrary: In the quoted case, Rex does not appear to be questioning anything. He appears to be dismissing the expression / definition in order to devalue Bach's approach.
It's not a very elevated sort of behavior at all. Questioning is a higher order of activity than dismissal. What Rex did (assuming Shrini described it accurately) is at best shallow rhetorical point-scoring.
Alan,
Like Gaurav, you also seem to suggest that questioning is better applied to our own thinking in terms of strategy, plan or questioning the decisions that went into design and development instead of applying it to software.
I am not sure if I understand that correctly. When I am questioning the software (not in literary sense) I am questioning everything that directly or indirectly connected with it. That includes our own thought process, decisions made during requirements, design and development, technology, business needs, user expectations, stakeholder expectations and so on - to various degrees of depth. I am not sure if separating the questioning software, questioning project context (decisions, test strategy) and questioning anything else that matters would be of real use. All these are intimately connected. I can not seem then as unique, distinct parts of work. Also during questioning the software (that is main subject of the investigation)a sapient tester would take a tour around the software and observer, model, analyze, infer and improvise possible entities that can potentially impact the software under test.
I am perfectly fine with Rex questioning the use of the words in the definition (words formulated into a definition by James Bach) - I am only responding to his query here. I hope I will get some response from Rex or his associates on this some time in the future. I am not sure if I am making this as big deal. Rex challenged the definition that came up during the discussion and I am providing a platform to take that discussion forward for those who are interested.
Michael Butler has pointed out an important thing. I might be describing what Rex said or meant "wrongly" or I might be putting the statements out of context. Since I did not get an opportunity to clarify what he meant, I am posting this with the disclaimer. I hope Rex or his associates correct me if I am wrong. I am all for a debate and discussion.
Shrini
Hey Shrini,
Shouldn't it be this way -
Testing software is questioning it's correctness?
Testing software is questioning it's correctness?
That's a part of it, but there's more. I'd suggest that it's more complete to think of questioning software as questioning its value (or possible threats to it).
Correctness is one dimension of value, but you can have a perfectly correct program that no one cares about, or that is hard to use, or that can't be localized, and so forth.
hi Ganadhar --
>>>Testing software is questioning it's correctness?
How do you know what is the correctness? According to whom? Note that we are not solving a mathematical problem (one that has only one correct answer most of the times). Yes, while questioning/probing software, tester do deliberate on relative/subjective notion of "correctness" as seen/felt by stakeholders. But that is a only a small part of the whole story.
Shrini
Do we question a software or do we test a software?
IMHO, we test a software. The "how" we test the software is based on questioning the various aspects of the software.
The result acheived from each test may or may not be what is expected.
The gap between expected and actual may be different in the views of Developers, BA, testers, technical support,et. al and of course "the user". (Which shade of red was the pillar? Dark red, Light red, was it actually red?)
"Questioning Software" in my opinion would be similiar to what Rex said...
Hello! Mr Software.. How are you?
Ahem!! We are not actually expecting a reply to the "question" from the software..
Are we?
Additional Note: How many definitions of Testing do we know?
Pickup any software testing book. some authors refer to other authors for the definition. Some write their own definition. (Sometimes explaining why)
Never the less, from SO MANY definitions to testing, what really matters in which one you subscribe to. They all achive the same goal? (ok.. Again "goal" can be a perception)
Questioning software or Questioning the thoughts to test the software = Common Goal.
You did a "philosophical" point very good and similar to Mr Rex Black
Post a Comment