Friday, March 06, 2009

C-DLICE’ing in software Testing

Let me take credit for making this pneumonic up: C-DLICE. I was listening to Michael Bolton's video interview on youtube. He said testing is more than verification, validation and confirmation. It is about Challenging claims, Discovery, Investigation, Learning and Exploring. Any skilled tester would do one or more of these activities as part of testing. By explicitly chaining them in a pneumonic, a tester can focus on a specific aspect of the interaction with the test subject.

Let me expand the pneumonic –

C – Confirmation. Other than traditional words like Verification and Validation (whatever may be the meanings of these terms) most people on this planet think that sole aim of doing testing is "confirmation". It is seen confirmation of claims made about the product, conformation of what developers "felt" that they created in response to requirement specifications that received and interpreted to their best of abilities. The confirmation about some specific user expectations (assumed to be routed through specifications into the software product). In basic form confirmation is somewhat like "Click this button, such and such thing should happen – Does it happen?" While confirmation is important aspect of testing, any testing that focuses on confirmation will become boring, brain dead and poor way to think about testing. Notions like anyone can do testing, process plays important role in testing, testing without test cases and requirements is not possible – are creation of confirmation oriented testing. I will not dwell upon challenges in confirmation oriented testing. And there is a big deal called "reference" against which you confirm – specifications. If your reference is wrong, ambiguous and incomplete – so will be your confirmation. That is the weakness of confirmatory testing.

Though my pneumonic is more about DLICE, I will still keep this "C" in there to remind us that confirmation may be as important as other letters in the pneumonic.

D- Discovery. While we test tester, we discover information about software, certain behaviors. It is like discovering an unknown island. As product grows bigger in size (in terms of codebase), discovery becomes important. No user would use the software as per the user manual. Discovering way in which software could be used and misused is important aspect in testing. Discovering is about finding information about unknown areas. For a growing software application, every time there is more to test than before – more to cover than before. Under such circumstances, you constantly discover the application, its variations, behaviors and so on.

L- Learning- This is a freaky one. A significant part of testing is implicitly spent on learning about everything that software under test. Be it business domain, technology domain, community of users using the software, cultural and social set up in the organization producing the software, we learn all time. Learn about how the software is constructed, deployed, distributed and so on. Often, I have seen people downplaying "learning" aspect of testing as they would like to position themselves as "experts".

I – Investigation As testers we investigate claims about the product. How people perceive the product? Investigate inconsistencies. Investigate bugs, Investigate impact of a new technology, software change on the over image of the software. Investigation is about focused information gathering, analysis on certain events. Examining the evidence etc. Investigation starts off as open ended.

C- Challenge (used as verb) – As testers we need to constantly challenge the assumptions, beliefs around how people think about software. What each stakeholder thinks about the capabilities of the software? Challenge the premises and so on. Challenging would require the design of tests, experiments etc to expose the weakness of an aspect of software.

E – Explore – Somewhat similar to Discovery, Exploration enables any information gathering exercise. Explore market conditions. Exploring is about taking a tour. Exploration helps in modeling the problem space. Exploration is more open ended than investigation.

Notice that each letter is has some overlap with others. You can learn while discovering or challenging a claim or exploring a feature. You can investigate something by exploring it or discovering it. You can challenge something by investigating it or learning about it and so on. One way to think about DLICE is – Discover like Magellan, Columbus, Learn like a learning a new language, Investigate like Sherlock Homes, Challenge like Lawyers, Explore like exploring moon's surface or deep African jungles.

Few practical themes to apply dlice'ing

  1. When there is a new thing most people around you know little about - something that you do not understand well , then – Discover, Explore and Learn
  2. When there is something that several others know but you do not – Learn through exploration, discovering
  3. When there is"suspense" or "mystery" about a thing – Investigate – a defect, a strange behavior etc.
  4. When there is some that is "well known" to you (you are pretty sure) about some claim – Challenge it and prove your point ( backed up by prior discovery, exploration, investigation and learning)

So, next time you feel bored doing testing, try switching your focus … try doing some investigation, discover new ways of using software or explore an area of software and so on. You would find that testing is always interesting but you were told about only one dimension of it (confirm, find bugs, check it passes tests) so you felt low or bored about doing testing that way ….




Anonymous said...

Hi Shrini - was wondering when you'd blog again.

I have no arguments with the premise, but do have two quick questions for you.

How do you pronounce C-DLICE - is it See-Dee-Lice, or is it sort of like "Slice" with a d shoved in there? I like pronouncing DLICE like DeLice - like I'm removing the Lice (the little bugs) from the software.

This is really more for Michael, but I'll ask you regarding his comment. I agree that testing is more than verification, validation, and confirmation, and have no problems with the cdlice part of it - BUT - I see verification, validation and confirmation more referring to the "what" of software testing, while C-DLICE refers to the "how". You can verify software (to an extent) by investigating, learning, and exploring the feature set.

VIJAY said...

Hello Shrini,

Once again a good post ..

I have few questions on this DLICE process, would you please answer them. It would be helpful in understanding them better.

As you said, some of the Task in D,L,I,C,E are very much overlapping in one way or the other.
For example When you discover(D) a product, you as well are learning(L) about it. I couldn't figure out the dominance of discovery over Learning. I feel if you learnt something successful which noboby has ever tried, it means you have discovered something. But intuitively learning is the process you would have followed.
So where would you draw a line between each of the tasks involved in DLICE ing. If DLICE is considered to be a process to be followed for Testing, it should have clear demarcations between the Tasks followed in it, right?

The following lines are specific about the 'D' part of DLICE ing:
I personally feel some help has to be extended for the Tester to follow this 'D'(discovery) part of DLICE ing:
1. Without the functional architect's help, the developers cannot discover the characteristic of the Business process they are adding or enhancing. And it follows like a chain reaction;
2. Without the Developers documenting what change they have done in the codebase(for code maintenance) and what features are tuned in the application(by means of Specifications), the testers cannot collect the information about the change and discover the so called unknown area..
I feel the 'C'(confirmation) process has to be carried out as sub-process in each of the stages i have enumerated. If the Architects, Developers don't confirm the correctness of the information they are letting out to the Developers, Testers respectively. Then it would cripple both Development and Testing Activity which is absolute waste of time and more importantly the cost.
How would this DLICE be used by a Tester if there are no co-operation from the Architects, and hence forward the Developers

Thanks for your time!