100 Questions or 100/10000 Test cases in 20 minutes …

That is what I would have named a blog post that my buddy Rapid software tester (context driven tester too) Pradeep Soundararajan wrote few days ago.

If every question is a test case or represents say 10 test cases, can you create 100-1000 test cases in say 20 minutes… Any James Bach student, context driven and or rapid tester will demonstrate that it is possible ….

Without any KA/KT, training, domain knowledge, no pages of documentation …? Plain vanilla testing at its best. Unbelievable right?

One might say --- that is cheating …!!! you may scream… that is not a test case. Where are steps? Where are the detailed information about application? Can a novice, low skilled tester use this? Can this be used for next five years? Can that be automated? And run in night when people are sleeping and give the results in the morning (in plain pass fail – way)?

The answer is “No”

Please note, while we create lots “frills” or decoration around real test and call it as test case – at its core a test is a question that we ask the program and program responds to it with one more answer. Some of these answers we can observe, assess, report while lot others go un-noticed. And while this happens, the environment or platform also responds to the question.

Questioning is a key attribute of a skilled tester. Questioning is also important aspect of learning. Unfortunately, right from our childhood (remember your father and mother saying “this kid is too much – asks too many questions. You will know when you grow up”!!!), through our education and now in the Job, Questioning is not encouraged.

Why?

• Questioning is considered as disrespect or contempt
• Questioning is indiscipline
• Questioning is disturbing
• Questioning is embarrassing when answer is not available
• Questioning sometime is considered as silly and stupid
• Intelligent people do not respond to silly questions
• Questioning at times make everyone think – that stalls the progress in some cases
• Questioning in a group is considered as bad
• Questioner is a labled as “trouble creater”
• Question = Trouble, more work, Road block
• Oh Gosh, we have not thought about this at all … this is terrible, what do we do now – is not easily coming response for a question

Today’s testers are forced to follow processes, documents, checklists and other “standard” things. If one were to follow the kind of thinking that Pradeep displayed – testing can happen with least information – a lot can happen in small time. I have heard testers who can not start testing (or test design) until they get specification, training and other supporting material. This is a damaging trend for testing profession.

As a famous punch-line of Café Coffee Day (popular chain of coffee joints in India) – “A lot can happen over coffee", goes, can I say “A lot of testing can happen in 20 minutes for any application”.

Just give me the *stuff* to test … I will flood you queries that can potentially lead (if investigated and answered) to an arsenal of information about application under test …

Shrini

Types of Equivalence: Equivalence Class Partitioning - II

Following this post of mine, I have been studying deep into understanding of this technique. Here are few more thoughts related to “equivalence”.

Here is ECP in nutshell - “Group a set of tests or data supplied for an application. Assert that all the tests/data belonging to group will teach you “same thing” (application behavior). Hence it is “sufficient” to use only one value/test from the group”.

Fundamental to ECP is the concept of “Equivalence”. Most of the authors or proponents of this technique give examples of date, integer fields and demonstrate identification of classes and equivalence. For example if you consider a date field in “NextDate” program, using “generally accepted rules” governing the usage of dates in Gregorian calendar – you can identify some classes – All the dates in the month of January can be considered as equivalent (except first and last day of January and first and last month of the century – which are boundaries). These “canned” classes appear to be applicable for every application that has date field in a “next date” function. Another example would be a field of integers (1-100) – most authors have mentioned the example of 2-99 as one equivalence class meaning all numbers in the range 2-99 would be treated “alike”.

I would call such equivalence that can be arrived without knowing anything about application, it logic and programmatic implementation details as “Universal equivalence”. It is easier to explain the concept of ECP using “universal equivalence” – date and integer fields are the most popular examples. But I see a danger here – the way ECP is explained using “universal equivalence” – it leaves out lots of key details such as basis for equivalence.

What are other forms of Equivalence?

Functional Logic equivalence - Consider the “Age (1-150)” field. Application logic might enforce that Age range 1-16 considered as one eq class (Kids) and others like 17-45 (Adults) and 60-99 (Senior Citizen). This kind of equivalence is very straight forward, easy to derive. Often specifications help us to arrive at such equivalence classes. Here is where the classic examples of “valid” and “invalid” EQ classes seems to have been originated.

If one were to go by pure functional logic equivalence, it would be sufficient to model Age parameter having three eq. classes and hence one value taken each from these classes (3 in all) would provide “complete” test coverage from an ECP perspective.
Dr Cem Kaner calls this as “specified equivalence”.

Implementation Equivalence – This is where one deep dives into how data is processed (validated, accepted, rejected), passed around (within application components) and eventually stored or discarded after use. Here we would talk about programming language (data types), software platform (OS and other related programs) and the hardware platform.

Dr Kaner identifies another two sets of equivalence - “Risk based” and “Subjective”. If the equivalence is in the eyes of tester (“these two tests appear to teach me same thing”), this form of equivalence is called “Subjective” equivalence. If a notion of equivalence is established targeting a specific class of risks or errors, it is referred as “risk based” equivalence.

Thus one way to apply ECP effectively is to start with universal equivalence and go on refining the sets EQ classes as we go deep into the application and platform (add, modify, delete the classes and their definitions). Implementation Equivalence seems to be the lowest or the last in the chain overrides the specifications of classes as determined by higher levels of equivalence (universal or functional logic type)

One question to spice up the discussion – Is ECP a black box technique?
Yes if we restrict to “universal and functional logic” equivalence.
No if we deep dive into code of the application and look around at platform (software and hardware)

What do you think?

[ Update ]

ECP attempts "simplify" a big picture (data domain with infinite set of possible values"). When attempting to apply ECP for a data variable, best starting point would be "what is that big picture I am trying simplify using ECP"? This is a top-down approach - model, understand, analyse, hypothesize the big picture then go to next level and then think about EQ. classes. I have people mostly approaching this from "bottom up" approach - think about valid and invalid classes first (or even actual values) then if possible think about the big picture.

Which approach you think is a useful one to start with?

BTW, there is "Equivalance principle" by Einstein related to theory of relativity. Can I say equivalence as applicable software tests is "relative" in nature?

Shrini