Reductionism and Test Techniques - who, what?

Scientific reductionism is an undeniably powerful tool, but it can mislead us too, especially when applied to something as complex as, on the one side, a food, and on the other, a human eater. It encourages us to take a mechanistic view of that transaction: put in this nutrient; get out that physiological result.
- Michael Pollan

I have heard statements like the ones below several times … I am sure many of you have.

- Orthogonal Array or pair wise technique reduces the number of test cases and hence can optimize the test effort

- Use of orthogonal array based testing (Test design technique) has demonstrated to produce superior test plans that improve testing productivity by a factor of 2.

- Equivalence class partitioning (ECP) is a functional testing technique that systematically reduces the number of tests from all the possible data inputs and/or outputs, and it provides a high degree of confidence that any other data in one particular subset will repeat the same result.

- Boundary value technique catches the bugs occurring near boundaries. Use of boundary value technique increases the effectiveness of test coverage.


What do you notice common in all these statements? Faith and reductionism. When you say “Technique XXX does this or help in this [reduce the number of test cases], you are forgetting that it is not the technique but it is YOU as someone who is making an assumption/assertion.

For example, when you say “Orthogonal array technique reduces the number test cases” – you actually saying, I as a tester using this technique making assumption that only pair wise test cases generated by this technique matter – rest I ignore.
Or
When some one says “ECP technique systematically reduces the number of test cases” – that is a view of reductionism. It is tester's hypothesis, judgment and assertion that certain sets of data values CAN be treated equivalently. ECP does not "reduce" ANYTHING by itself systematically or otherwise. ECP's principle per se is that there are groups of data that can be modeled to be treated identically by the AUT

I can say that, this is a real difference. This shifts the focus from the technique to the person who is using it. You then can not shift the blame to the technique if it fails.

As testers we use lots of tools, techniques and heuristics to understand, operate and observe the test objects. Every time I use a technique or a heuristic, It is my explicit choice and I make a set of assumptions and assertions. The goodness and applicability of results of the test or technique, directly depends upon these assumptions and assertions. Technique/tool has no role in it.

Pradeep Soundararajan and Michael Bolton have it here – A focus on tester not on test or a tool or technique…

http://www.developsense.com/2007/09/if-test-passes-in-forest-and-no-one.html

If you take a gun and kill a person – what do you say? Gun killed the person?

Shrini

Percipient Subject and Software Quality

Jerry Weinberg in his famous book “An Introduction to General Systems Thinking” mentions quoting Albert Einstein…

“Belief in external world independent of percipient subject is foundation of all science”

Let me extend this to Quality (or Software Quality) …

“Belief in external world independent of percipient subject is essential for the notion of Quality in software world”

All definitions of Quality (except that of Jerry) from Juran, Deming and others somehow assumed the existence of “external world independent of percipient subject”. Hence Quality has been more or less defined in absolute terms.

Let me remind you -
- Quality is value to some person (one who matters)
- Quality is not an attribute of a *thing” – we can not measure Quality but can only assess it in “relative terms”
- Quality is in beholder’s perceptions.
- Quality is not an intrinsic attribute of any “thing”
(Quotes from Jerry, Michael Bolton and others…)

Bonus Tip:

“Belief in external world independent of percipient subject is essential for the success of a metrics program”

Most of Metrics programs that are claimed to be successful stand on this belief.

Howzzzzzzzat?

Shrini