Chocolate and Prayer - An Anti Pattern for BDD

In a school, for first graders, there was a practice or ritual in the morning every day. The kids used to assemble and sit in a designated place as they come in. They needed to say a prayer with closed eyes. When they finished the prayer, each kid would find a chocolate bar in front of her. The kids would happily take it, eat and proceed to their classes. This ritual ran for several years. Kids thought that chocolate is prize that they earn for saying prayers and none questioned the ritual. Years passed by. The length of prayer became smaller, kids got their share of prize - chocolate nonetheless.  On one day - kids assembled in their usual place and were preparing to say prayer - they saw chocolate bars in front of each of them - already. With none around  - few kids took initiative and grabbed chocolate while few sincere ones proceeded with prayer as usual.  After few days - following law of diminishing returns, these sincere kids to started to skip prayer and focused only on eating chocolate.
After several years of this ritual - one curious kid, unable to control his thought about why they get a chocolate everyday in the morning (note - prayer is long forgotten), asked his friend. "None knows why, my elder brother tells me that there used be some prayer before they got their chocolate" said the friend not so interested in the question.

Now, imagine this is a multi year social experiment conducted by school authorities in collaboration with educationists - what would you infer ? You might say, initially kids got their prize after prayer (a good and recommended activity to start the day in school) and when chocolate was given prior to any prayer, kids simply forgot or dropped the idea of prayer. Economists would call this as "incentive" to elicit a specific behavior from a group of people.

Let us come to our world and let us try to map prayer and chocolate to BDD (behavior driven development) and automation. As original proponents of BDD wanted it solve certain problems and automation apparently came out as chocolate, prize that follows doing BDD.

As I understand  - BDD was intended to bring business analysts into the party, develop a common vocabulary between Dev, BA, Testing and stakeholders and address some of the perceived problems close cousin of BDD - the TDD, test driven development. Dan North explains the background and history of how he landed with the idea of BDD. As Dan narrates - the practice of BDD proposes to focus on the behavior (change from keyword "Test"or "requirement") software should demonstrate for a feature that client wants. In order to develop a common vocabulary - BDD needed to restrict the representation of this behavior using a set of keywords and the behavior required to be in a non technical language (remember they needed to bring BA's that are non technical into the party). Thus using a class of languages (meta language, I guess) like Gherkin which is a type of DSL (domain specific language) BDD ushered a practice where intended software behavior and corresponding scenario or an example was represented in a format like the one below


As [Role/Stakeholder]
I want to [A feature or behavior]
So that [business outcome that is worth paying for]

Scenario
Given [Initial or Preconditions]
When [ Action performed to invoke the feature]
Then [Expected result that software needs to demonstrate]

As Liz Keogh, one of early collaborators with Dan on BDD development, says -key challenge BDD was intended (broadly among other things) to solve is facilitate and improve communication, discussion and debates about what the behavior should be,among developers, testers, business analysts and stakeholders.

That was a prayer  - BDD's objective for effective communication.

After looking format of BDD scenario/user story, full of keywords - a smart developer would have thought "I can parse this and generate a skeleton code which can be implemented later as automated test". This is that chocolate that was promised to everyone in the team. Thus a strong distraction for original objective of BDD was born in the form of automated tests out of BDD story/scenario.

The theme of automation attached to BDD become so powerful with loads of frameworks such as jBehave, Cucumber and others overshadowed everything related BDD. At some point of time, doing BDD meant using jBehave or cuccumber and creating automated tests.

The power distraction of automation (chocolate from our story) instantly hijacked communication/discussion about behavior (prayer) and practitioners BDD started doing only automation. This is the anti pattern that I wanted to highlight in this post. I have seen several instances where testers/developers/BA's were worried only about which tool or framework to use for BDD and which automation framework/library to use. The stakeholders on their part were sold on the idea that they would get "Executable specifications that come with dual benefit - representation of behavior and automated test". They could not ask more.

Alas, in the process, BA's, testers and developers instead of sitting together and discussing about what "Given" should lead to what "Then" or what "When" leads to what "Then's" - sat in silos and happily created loads of BDD stories and some tester or developer jumped straight away to implement automation.

I am not complaining about automation that is embedded in BDD per se - I would like people to reinstate the prayer - the focus on cross function collaboration, you can have your chocolate (automation) anyway.

Time to read Dan's post on introduction to BDD and also posts from Liz on the aspect of communication ?

Sure Ways to Reduce Test Cycle Time through Automation

World is simple, we complicate the world for the sake of it – says a friend. There is a simple concept called "automation" and another one "testing cycle time". Why these terms can not be simply understood without much fuss and inquiry? he often argued with me. This friend is a manager and works for IT services company. A constant pouncing on me by this topic of test-cycle-time-reduction-through-automation and related hype, frustration and feeling of achievement, made me to think deep about basic laws or golden principles that govern this phenomenon of cycle-time-reduction.For the benefit of my blog readers, here, I make them public. Read them, understand them, implement them and be blessed. A caution here any criticism and cynicism about these laws will have harmful consequences to the beholder. These are golden principles and axiomatic about test automation!!!!

First principle - “About Testing”: Strongly believe that software testing is a deterministic, highly repeatable and structured process – somewhat akin to a step-by-step procedure to produce say a burger or car. You have to believe that given a fixed scope of testing, it always takes a finite and fixed time (effort) to complete testing. Needless to say, you have to have absolute faith in testing processes and standards. Your faith in the power of processes and standards making testing predictable and repeatable is an important success factor. It is also necessary to abandon any misconceptions you might have about relationship (or dependency) between testing and automation. You should treat testing truly as a mechanical, step-by-step and repeatable process to ensure Quality. Positive ideas about metrics to improve testing and consistency are - a sure bonus. You should resist and fiercely oppose any attempts to link automation and testing, claiming that automation can and should work independently.

Second principle – “Definition and Meaning of testing cycle”: Never challenge or probe definitions and meanings of word “Testing cycle”. Keep it (testing cycle time) loosely defined so that you can flip in any direction when confronted by a skeptic challenging your claim of cycle time reduction. The more the vagueness of the term “testing cycle and cycle time”, higher are the chances of meeting goal of reduction of cycle time through automation. Any variables and factors that make “testing cycle” somewhat unclear – should be ignored and not discussed. It is important to have faith in the fact that “testing cycle time” is universally known term and does not need to be redefined in any context. You would be looked upon as genius if you simply talk about “cycle time reduction” and omit the unnecessary qualifier “testing” or “test” (as in “test cycle time”).

Third principle - “Playing to the gallery”: Use words like “business needs”, “business outcome”, “business processes”, “success through business alignment”, “Time to market” etc., liberally during all communications related automation and testing cycles. Confront any opposition by skeptics about automation and its connection to cycle time by “calling authority” – say “Business/Market needs it”. Strongly believe in statements like “Automation will help releasing products and services faster – and hence will improve customer satisfaction and company bottom-line”. Your success in achieving cycle time reduction depends upon how often and how strongly you make reference to “business” and “business needs”. These powerful words that do all the magic required. You need right rhetoric to spread the message. Another important keyword here is “market”. Make sure you thoroughly mix up and use the terms “Testing cycle time” and “Time to market” interchangeably. The more you talk about “Time to market” and how automation can directly help it – more you look authentic and convincing.

Forth principle - “Motivations and Incentives”: Believe economists when they say “incentives” drive change and motivate people. How can this not work here? The last but not least of the measure that one needs to be aware of is to provide incentives for people to reduce cycle time through automation. Define performance goals of individuals involved in the automation in terms of cycle time they reduce through automation. Penalize those that question the idea and fail to meet the performance goals. The automation initiatives tend to be successful in their stated goals of cycle time reduction when they are integrated with performance goals of individuals involved in the game – especially automation team. Also make sure (when automation is done by a decentralized team) to define and impose the performance goals ONLY on automation team and suppress any attempts to include manual testing team. After all automation and testing are not related in anyway and it is the responsibility of automation team to bring about cycle time reduction. Right?

If you find these principles and ideas rather strange and are intrigued – one of the several possibilities is that you might be working in a software product organization as opposed to IT and IT services organization. The folks in software product organizations that build software – unfortunately approach automation some what “differently” and cycle time might not be a very familiar term to you.

If you are not from software Product Company and still confused about the ideas in post and you think they are inconsistent or incoherent with each other – do comment. That is a good sign that you are thinking about the topic.

Let me know if there are other ideas and principles that you might have successfully used to reap benefits of cycle time reduction through (and only through) automation – I would be more than happy to incorporate them with due credit.

Shrini

Book Review: Selenium 1.0 Testing Tools - Part 1

In a first of its kind request, few months back someone requested me to do a book review – that too a book on “Selenium”. I wanted to learn Selenium since Long time, to compare and contrast traditional GUI based automation that I primarily worked on.

That is how I picked up this book by David Burns – a senior software engineer in test at Mozilla Corporation (Not Mozilla Foundation) I am not an expert in Selenium – this review of mine as from the eyes of a learner. I have attempted to convey how someone like me trying to learn Selenium would find this book. Those who are with Agile movement and likes might find my comments rather unusual – bear with me. I am from a different world …

Where do I start from? Before I started reading the book – I did some reading on the web. I was looking for a good definition for Selenium. Adam Goucher defined it as “Selenium is, at its core, a set of tools which let you control a browser. What you do with that control is completely up to you. Some frequent fliers use it to reserve aisle seats on their flights, but the majority use it as part of their testing process. After all, the end user of your product is not going to be experiencing the product through their browser as well, not via some wire-line unit testing framework.” I think, Adam’s definition is simply the best one to get started.

I would strongly recommend anyone starting with Selenium to read its history and especially support from Google.

As it happens with all most all books on “selenium” – even this book claims that Selenium tests the application and not selenium is a test automation framework. To me “testing” and “automation” are two different but complementary things and I think Selenium as an Automation tool rather than a “testing” framework.

Here is an account of my experience with the book – chapter by chapter.

- The book starts off with IDE installation and Selenium- “Hello World” kind of test (record-playback). The chapter covers few additional items to sample tests such as adding comments, debugging tests, working with multiple windows. AJAX applications make a sudden appearance in the chapter - not sure why. I feel the section on AJAX is not well connected with rest of the chapter. The section on “Saving Tests” is very brief – could have been expanded to cover exporting or converting IDE tests in other languages (Java) and automation frameworks. A brief introduction of TDD or ATDD or kind of automation that Selenium supports would have been a good way to start a book. Explanation of what is “Selenium” lacks depth and can confuse beginner and experience a like.

- The chapter on “locators” helps the reader to learn about locators. Coverage on DOM, XPath, CSS is pretty good. I would have liked the introduction be deeper. Finding the elements that make up a webpage is the work of these locators. You can locate elements by ID, name, DOM (through java script), XPath and CSS selectors. If you read the introduction of the chapter and summary – I am still confused about this question – “When we can get the page elements through record and playback function – why one needs to bother about locators – so many of them”. I think I know the answer. But I would have expected the explanation to be part of the chapter… “Why one needs to learn about Locators”. If you don’t ask this question – you can very easily follow the chapter and do hands on exercises without any problem and learn about IDE and some tricks

- As it is the case with other chapters so far – the introduction of chapter “Pattern Matching” lacked the depth and connection. It does not seem introduce the chapter by attempting to answer “Why pattern matching? Where? How this is applicable to automation in selenium”. The chapter, as in others mentions “In this chapter we shall do …..” I would have expected the book to cover – why part of these activities. I would have liked to see “how these activities fit in overall landscape”. Usage of pattern matching in general has been explained well. Use of “exact”, “glob” and other nuances of regular expressions, is covered in detail.

I will cover in next posts remain chapters of the book … Do Check back

10 Commandments for Test Automation Outsourcing

Most of my test automation experience and learning comes from working with IT groups and GUI based automation using COTS tools. Outsourcing of testing is a top agenda on today’s IT manager and automation happens to be one of the popular and “most frequently discussed” item.

I am regularly asked to formulate, present and consult automation projects in IT space. Based on this experience, I am attempting to formulate commandments - about 10 of them. I believe these are few considerations that an IT manager who is thinking about outsourcing test automation work.

Automation is not an answer to your testing problems such as limited testing bandwidth, limited time to test, poor application quality etc. These are true testing problems – one way to address them is to set your “manual testing” right then think of Automation. If your manual testing is poor, throwing automation in that will only cause that “poor” testing be completed quickly.

Automation is White Elephant While automation has clear benefits only when it is treated carefully. Automation is not a turnkey, as your application undergoes changes; automation solution needs to be maintained. Plus there are recurring tool costs, training costs and test case/project management over heads. Is your vendor telling about this sufficiently? Is your vendor downplaying this aspect? Watch out …

Anything that is quickly creatable – is quickly perishable Do not believe claims of “script creation in minutes”, “Automated script generation” etc. Any enterprise level serious automation is software development and requires clear attention and methodical approach to design/create and maintain.

Your business users can not create and own Automation solutions Let your business users/SME’s and domain experts do what they are best at – business support. Do not believe claims of business users creating and maintaining automation solutions. It requires good amount of testing and automation knowledge.

Judge your vendor by the questions they ask about automation Suggesting an automation solution requires thorough study of context of project, release cycles, current state of project, future expected life of application, business expectations, state of automation practice etc. Vendors who propose a “standard”, “gold plated” automation solution without asking questions to probe the context are “simply” selling you something – be skeptical about such vendors.
Higher % of offshoring in automation higher investments to make While moving work to offshore will give cost benefits, when it comes to automation there are number of factors to be considered while deciding how much of automation can happen at offshore. Key thing is about creation of application environment at offshore. If a local application environment can be created at offshore with tool licenses – higher degree of automation work can happen at offshore. It is important to note that for GUI centric/COTS automation tool based automation, both automation tool and application should reside on same machine. Hence pay attention to your application environment and feasibility of creating local environment at vendors offshore location before deciding the amount of work that can move to offshore. Even when such local environment is available at offshore, certain activities like acceptance testing, demo to users, and certain types of test cases will need to be done only at onsite. Be sensitive to these factors. Make sure your vendor asks about this and suggests suitable alternatives.

Pay attention to dependencies and Quality of current test Artifacts A successful outsourced automation requires that project dependencies are well understood by all stakeholders. Access of application from offshore, access to testers/developers/SMEs on test cases, Access to reviewers and code acceptance people are few dependencies that one needs to track as part of project. If automation is on the basis of existing manual test cases- make sure that these are detailed enough and available in a form that can be sent across to vendor offshore team.

Decide Acceptance criteria Formulation of Acceptance criteria is not item that is often given least importance while planning outsourced automation projects. Identify an internal owner in your organization who will accept the code/solution delivered by the vendor. Make sure that this person gets engaged in the project from the beginning and formulates the acceptance criteria along with vendor technical team. Failure to identify automation acceptance person from your end and getting a formal agreement on acceptance criteria can leave you “High and Dry” and leaves an open space for the vendor to deliver any “working” automation code but not necessarily the one that stays for long time.

Avoid linking an automation project (and its deliverables) to application release dates one common mistake committed by IT groups outsourcing automation is to link the automation deliverables to immediate ensuing product/application release and cut down time/effort for manual testing. One IT manager said to his project team “We are having a major release for application XXX in January and we have automation solution coming from a vendor by December. Since Vendor has promised that 90% of manual testing will be automated – let us plan to allocate 2 weeks of testing instead of 10 weeks planned earlier. As promised by vendor we can deploy automation and cut down testing by more than 50%”. What is the problem here? What if Automation delivery slips for the reasons beyond control of everyone? What if development delays? What if due to poor test case quality and fast changing application, automation scripts give inconsistent results? This will result in conflict - “believe automation” or “get some good manual testing done”? Such conflicts can severely impact your releases and hence business plans.

Record and Playback (RP) Automation is for Kids This one point can never be overemphasized. Many IT managers still feel that record playback features of industry standard automation tools can help them to create automation quickly and thereafter their own resources should be able to record and create/maintain automation scripts. However, the experience has shown time and again that RP approach is not beyond learning about how automation tool works with application and can not be used for real time, sustainable automation. If vendor proposes this as a part of the solution – you should be alert and suspect the abilities of this vendor to deliver automation solution

Do you agree with these commandments? Any different experience?

[update]
Some additional tips :
What a Vendor should ask you
- questions about your manual testing practice
- Your objectives of Automation and expectations
- Readiness to take up automation in terms of test cases, application state and environment
- Ask about your expectations on ROI

What vendor should suggest you
- Plan for automation maintenance when supplier is gone.
- Automation may or may not reduce cycle time – that depends upon nature of tests, application technology stack, nature of tool etc
- Automation may or may not reduce the cost of testing.

What to look for in an automation proposal
- Acceptance criteria
- Automation design details – how tests will be structured
- Environment related assumptions
- Pre-requisites about tool licenses, test cases, test data, access to developers, testers and business users (for clarifications about test cases)

You can not automate testing – all of testing as testing is intrinsically a human thinking and investigation activity. What you in reality claim to automate is some portions of testing – namely “test execution” of some select test cases. Note that activities like test design, bug investigation and logging, Test results verification etc are still to be done by a human tester. Automation will take you some places but not all. Unfortunately, the places where it does not take you – are the ones where real problems lie – only a skilled human tester can help.

Shrini

A catalogue of Test Automation Benefits ...

Other day, some one asked me “in your opinion, what are the real benefits of automation”. That triggered ideas for me to write this post… Let me attempt to consolidate and list all the benefits people claim around “automation”

Real: In my opinion, these are “real” and achievable benefits.

- Consistent and Accurate Test results (nearly free from human errors) – when accuracy and accurate results are important? – Numerical calculations
- Untiring and can be run for long hours without any loss of efficiency
- Quick – No think time while running tests. (Computer program does not think – So not useful in those cases where you need to think as you execute. How does automation program help you to gain speed when thinking is required?)
- Supplements human ability to spot software bugs
- Helps to run big number of data combinations – can test robustness
- Helps in multi platform combinations (OS/browser/database and other application setups)
- Repeat the testing (test execution) done for configuration A in Configuration B also.
- Hence increased Test coverage
- Following non test execution tasks
- Generate and manage special set of test data
- Large Volume Test comparisons
- Automated workflows and alerts
- Environment setup

Fake: These benefits are on transition point – where the focus slowly starts drifting from “real” to “imaginary” and hence “fake”. These are false promises that are realistically not possible.

- Improvement in application quality (automation can not improve application quality – even testing can not … only developers and business analysts can)
- Improvement in Test process (Test process is pre-requisite for automation)
- Non technical people can do automation – no programming language required
Improved test planning (not sure how)
- By executing a set of test cases for a specified number of times – ROI from automation can be realized. Say in 10 executions –automation test pays for itself after that is “cost saving” all along


Conditional: These are the benefits that are realizable and could be reasonable but under a very strict set of conditions – “once in a while” cases. When these benefits are stated, mostly the conditions that must be fulfilled to realize these benefits, are not stated. This makes these benefits look as though there are real and universally applicable

- Simply – saves time.
- Test effort reduction
- Improved Time to market
- Improved test productivity
- 24x7 testing possible
- Knowledge retention


Totally outrageous: There are some really outrageous and are more or less like a typical sales pitch. People, who make and believe such claims, do not seem to understand human side of testing, testing itself and automation. They are just believe that machines are better than human testers.

- An automation test execution is equal to an hour of skilled human testing
- Automation can replicate human interactions
- Reduced dependency on human testers
- Solves problem of resource crunch and less time available for testing
- Reduced defect rate – fewer defects


Anything I missed?

Can Automation reduce cycle time or improve time to market?

Continuing on this discussion on test effort and manual testing – there is another popular variation in which Automation tool vendors claim “improved time to market” and/or “reduced cycle time”. In this post, let me dissect these claims and see how true and credible is this claim.

First of all let freeze what one mean by “Time to market” and “Cycle time”. Let me define the terms as bellow in the context of a traditional/Waterfall model type of software development.

Time to market is a time window between the time your start development (requirements and so on) till you ship the product in market for general public consumption. Depending upon whether you are doing a major release or minor release, this window may span from few months to few years (as in the case of Windows VISTA)

Cycle (when used without any qualification indicates a cycle of development and Testing) time is a time window for a software product under development. The cycle time can be divided into Development time and testing time. A development cycle starts with deliberation of requirements, design of features to be implemented. A test cycle starts with a development team “releasing” the code for testing to begin and test cycle ends with test team completing the planned testing for that cycle. During this period development team can fix the bugs reported. Hence for all practical purposes, a cycle time implies a time window between start of design/requirements until test team completing the testing/development team is ready to release next build.

So it is apparent from above definitions that cycle time is a subset of time to market.


Automation can reduce cycle time (less of a problem) only if (check how many items or situations that “automation” can control)

• Automated tests run without reporting ANY bugs on the software (A bug reported by automation means – some investigation and confirmation by manual execution that the bug reported by automation is indeed a bug with the application)
• Automated tests DO NOT report any run time errors (A runtime error by a automated test means some investigation and re-run.)
• Development team INSTANTLY fixes any bugs reported by Automation and these fixes are so well done that there will be no further verification (manual or automated) required
  Manual testing (a very small portion indeed) that happens in the cycle does not report any bugs. All the manual tests pass.
• If manual testing reports some bugs those bugs will be fixed INSTANTLY without requiring any further verificationBug reporting time, triage, investigation (if any) is so small that they are negligible.
  

Automation can reduce/Improve time to Market only if –

• All the items mentioned under “Cycle time” and
• Business stakeholders do not worry about outstanding bugs. They take the decision to ship the product as soon as Automation test cycle is completed (because automation cycle is NOT expected to report any bugs). So the end of the automation test cycle, shipping the product is a logical thing to follow.

If you analyze these situations, you would notice that many of the factors that influence cycle time or time to market are not under the control of “test automation”. These factors are to do with development team, quality of the code, quality of requirements, number and nature of the bugs reported by both manual and automated test execution and above stakeholder’s decision about those bugs reported. One can not claim that there will be cycle time reduction or improved time to marked JUST because x% of test cases are automated. A big and unrealistic generalization - only an automation tool vendor can afford to make.

So next time when someone says automation reduces time (either cycle time or time to market) – do quiz them and say “What do you mean”?

Bonus Question : Can automation accelerate your Testing? If yes, under what circumstances?

Next Post : Can automation address the IT problem of Limited (human) resoures and tight deadlines?

Shrini

When method dictates Goal – See Goal displacement in action - II

I wrote about Goal displacement part I here. Surprisingly .. no comments yet ... :(

Here is example # 2

Software Testing Certifications: A conversation between a certification enthusiast (CE) and a critique (CC – that is me)

CE : Do you know about this certification for software testing, this is very popular in Europe and US?
CC: Yes, I do but I am not sure if this really helps in evaluating skills of our testers.

CE: I think it does. I have heard about exam it seems pretty exhaustive and coves all aspects of testing.
CC: Umm … what do you think the certification is testing? What and how does it evaluate testing skill of the candidate?

CE: Certification tests the “knowledge” of the tester, familiarity to terms, definitions and experience of the candidate…
CC : Really? How? How a certification tests knowledge and experience of the tester?

CE : By carefully selected questions and evaluation by testing experts. The certification exam is based on a body of knowledge and questions – mostly objective type. I think one can rely on the exam.
CC : So … you are saying an objective (yes/no and multiple choice) type question paper is used to test the skill and experience of a tester. Does the exam allows theory type of questions? Can the tester debate on an issue? Does the exam involve any kind of putting the tester into real testing situation? Does it observe the tester in action ?

CE : Come’on, how it is possible? Certification bodies have their limitations, they can not set up “practical exams” to watch tester doing testing and then rate. Do you want certification body to set up a audio/video facility to allow for debates, questioning and real time situation simulation?
CC : Don’t you think, that you would right and reasonable way to assess the skill of a tester not using a fixed set of questions that emphasizes on memory recall, reproduction of text of study material?

CE: That is right … Look at feasibility of having such exams … what about cost of administering such tests? What about evaluation? It would be costly. That is why the certification bodies might have created a scheme of tests that are easier to evaluate and conduct on mass scale. This will enable a relative cheaper exam and allow more people to take the exam Right?
CC: Well …good point. But what is the goal of certification exams? What they attempt to achieve?

CE: Evaluate and Assess skills and experience of a software tester.
CC: But your current exam seems to be structured in such way that it is easier to administer and evaluate.

CE : Ummm… that is correct.
CC: See this is what I call as Goal displacement. Certification bodies wanted to design, administer and evaluate a system of exam to evaluate and assess skill of a tester. But they seem to have taken the path of designing an exam system that is easier to administer.
The goal of software testing certifications is to act as a mechanism of evaluating skills in software testing and provide a benchmark for the talent in that space. As the popularity of such certifications grow, so grows the need for system for mass administration and evaluation. This causes the change in the examination pattern and evaluation mechanism to facilitate the mass administration. Any part of the exam that is good from skill evaluation perspective will be dumped if that does not lend itself to easier evaluation of exam results.

Side Note:

Software estimation models (especially test estimation models) often suffer from Goal displacement problems. Other day I told my colleage " The reason why we struggle with test estimates is that we use simplified model of testing while estimating but reality is different hence estimates are typically go wrong". For that he said -- "OK .... we know that actual testing models are complex (non linear and involve critical thinking/Questioning etc). That is why we use simplified models of testing ".... How strange ..!!!!

This is another example of goal displacement right away ... To make estimation possible - change the testing model itself ... use a simple one. So the goal of using a good and resonable testing model that happens to be complex (and does not lend itself that easily to estimation) is replaced by a goal of finding a testing model that is simple and easier to estimate ...

what should be your goal - good testing or estimation?

Shrini