Here is a heuristic - "All formal/written requirements are vague – only exception is when requirements are collected using a process that does not involve any human”
(Wait … if at all such process exists, where it came from? There must have been a human being such a process needs to be designed by a human … …hence there is a human element here too … Is above heuristic, an axiom or hypothesis or law?)
So there will be always gaps between "Customers view", "end user view" and programmers view". ET can help you to explore these gaps.
Jerry Weinberg in his book on Exploring requirements.. - gives the example of a nursery rhyme "Mary had a little lamb" and goes on to prove that there can be at least more than a dozen ways of interpreting above sentence ... Remember we typically deal with requirements documents of 50-60 pages ... Imagine the amount of misinterpretation possibilities....
I was discussing with this with Michael Bolton, recently. I said “I can say for sure that as long as requirements are written in English, they will be ambiguous. English is a funny language". He immediately shot back and said "you can say same thing with Hind, Tamil, French or any other human language"... I laughed and said “well, you are right, how about this ... One way to get unambiguous requirements is to express them formal languages like computer programming languages ... say Java. "Class MyClass" in java is nearly unambiguous". Michael continued "Here, the ambiguities come, not from written requirement but from the process of translating what is there in one’s mind to formal language".
That brought us to the real truth about software requirements ... "As long as one human tries convert what is there in her mind to some thing that written form; mind to document or mind to mind data transfer .... There will be ambiguities"
People can say a thing in different words but might mean one thing or they might say one thing and might mean differently....
Welcome to the world of software ... a discipline with high human content ....hence we need skilled software testers to keep world "lighted"
First thing you need to acknowledge is Requirements are fallible - they can be wrong - horribly incorrect (so will be corresponding program). If you base your entire testing only on requirement - you can go wrong by miles ... beware of this requirement based testing trap...
BTW there is a type of testing called requirement based testing ... can you imagine how narrow it can be?
You might have heard of people counting requirements … Can we count requirements as we count apples and pizzas?
What do you say about this concept of Requirements Traceability Matrix (RTM) – a holy thing that most swear by? Fundamental principle on which the concept is that “requirements and test cases can be counted”
Please… for God’s sake understand requirements, bugs, test cases are not real things and can not be counted in a way you count physical things, they are mere concepts, ideas …
6 comments:
Great essay, much needed. I'd like to add a couple of points.
First, people will have an easier time remembering this important lesson if you don't say "requirements" when you mean "requirements documents." People require what they require, and that may be ambiguous or not. But when they try to describe what they require--that is, produce a requirements document--they will always produce ambiguities, as you describe.
Second, there are cases where the requirements document may not be ambiguous, as when produced by a machine. I had a student who made a business out of converting COBOL programs into COBOL programs that ran under a different OS or H/W or compiler. The requirements document we unambiguous: "The new program must behave exactly like the old program in every case."
So, in order to test by requirement, all you have to do is run every possible COBOL program against the new system and compare what it does. That's not ambiguous, but it's impossible. So, even if you do manage to get an unambiguous requirements document, you couldn't really test against it anyway, so there's no sense talking about such a beast.
t our AYE Conference, we have a number of sessions dealing with this testing lie and related problems.
I don't know. I think you can count bugs in certain situations. For instance, I keep track of the number of bugs reported by customers once the product is released, where in the product they cluster and how long it was before the bug(s) came in.
This helps me learn where my coverage was not adequate (why) and my how how much (quickness of report).
Also, some environments (low level telco springs to mind) have very detailed requirements with relaly not that much wiggle room. (The protocol is this, and only this.) So perhaps you could count requirements there as "real" things.
-adam
Good post, Shrini.
Another thing to remember is that what the client wants, and what the client asks for, are two seperate and distinct things. The client will also want things they don't ask for, or aren't in the requirements document. Just to take a perhaps overly simple example, noone ever specifies that text must be readable on a webpage. It goes without saying. Once you acknowledge that there are things that go without saying, then you cannot possibly hope to achieve quality based purely on what the client is saying.
This is why I try to reserve the word "requirement" for items that are detailed in legally enforceable contracts.
It's also why I'm a huge fan of legitimate user acceptance testing with real users using the application however they use the application, under realistic conditions. Feedback from that kind of testing will certainly point out where the "discrepancies" in the requirements documents.
Better still is to do the same thing very early on with a sophisticated prototype.
Very insightful Shrini.
--
Scott Barber
CTO, PerfTestPlus, Inc.
Executive Director, AST
Co-Author, Performance Testing Guidance for Web Applications
"If you can see it in your mind...
you will find it in your life."
Something else to consider is that the software development process involves educating the customer on how they can help with requirements gathering. There are a number of critical elements that the customer has control over that can make or break the success of gathering accurate requirements. We have a two-part series on this topic that you might be interested in: Requirements Capture: 10 Keys to a Successful Software Development Project - Part 1 and Requirements Capture: 10 Keys to a Successful Software Development Project - Part 2.
Post a Comment