Differentiating between bug and feature - more often than not is a result of someone (typically a developer or a tester) trying to prove some other person (typically again a developer or a tester)wrong. Somebody says "See this seems to be a bug to me - I trying to be like typical end user" where as somebody yells back "Look this is as per this document and no where it is mentioned that the feature should work like this".
In simple words, the difference between bug and a feature OR "Desirable" and "Undesirable" OR "Expected" an "Not Expected” - is with respect to some REFERENCE. What is that reference? Who defined it? How credible is it? The moment all the concerned parties and entities involved in Bug-or-Feature conflict agree upon this Reference - the distinction becomes very very clear.
In most of the cases - requirements document, market survey or some expert opinion is considered to be the reference. The confusion in most of times is because of lack of reference (Oracle) and still worse the lack of knowledge that “there is no Oracle".
If I find myself in such situation - I simply say "This is my observation on this feature. I think we should analyze and explore to see if anything wrong here. I would be happy to participate in this exploration. I would not get into issue of whether it is a bug or feature - I am not the right person to decide that. I only report on observations that I make with respect the features I test.
so next time when you hear an argument like this --- Just ask this question - "Can all of us first agree upon the reference?". I bet, you would sound lot intelligent in the crowd ....
Shrini
Monday, September 25, 2006
Tuesday, September 19, 2006
Are you making most of Test automation?
Test automation is a beautiful and a very handy concept. Making computer doing things while you watch all those things that you would otherwise don’t observe – can be a very powerful thing in Testing. Deploying Automation can magnify the reach of manual tests in lots of ways. Unfortunately, Automation in lots of places has been deployed with intention to replace (not supplement, enhance or extend) human Testing.
If you are using Test automation for following cases – you are making the BEST use the investment made in creating and maintaining automation suites …
1. Those tests that have a high chance of human errors, those features that under go minute changes frequently that a human eye is likely to miss.
2. Another extreme of (1) – Routine and mundane tasks like installation and smoke tests.
3. Frequently repeated, High volume repetitive Tests
4. For covering multiple platforms
5. Those cases where it would impossible to perform a test manually
a. Simulating some behavior that exposes a risk e.g. memory leakage
Less desirable or efficient Usages of Automation
1. Automating a set of low power manual tests just to cut down the cycle time of testing of those tests manually – without evaluation of those tests.
2. Automating every test that is possible to automate and aim at covering maximum of testing by automation – a notion of “more is better”
Shrini
If you are using Test automation for following cases – you are making the BEST use the investment made in creating and maintaining automation suites …
1. Those tests that have a high chance of human errors, those features that under go minute changes frequently that a human eye is likely to miss.
2. Another extreme of (1) – Routine and mundane tasks like installation and smoke tests.
3. Frequently repeated, High volume repetitive Tests
4. For covering multiple platforms
5. Those cases where it would impossible to perform a test manually
a. Simulating some behavior that exposes a risk e.g. memory leakage
Less desirable or efficient Usages of Automation
1. Automating a set of low power manual tests just to cut down the cycle time of testing of those tests manually – without evaluation of those tests.
2. Automating every test that is possible to automate and aim at covering maximum of testing by automation – a notion of “more is better”
Shrini
Tuesday, September 12, 2006
Top burning issues in Software Testing ...
Top 10 Burning issues in Software testing ….
While thinking about asking a thought provoking question to make James Bach - write about it, I got this idea. Here is a post that lists top 7 (want to reach 10) burning issues in software testing today. I solicit other issues and thoughts about how to address them … Write to me, I will consolidate them post back on this blog …
1. External Issues
a. Business pressure (cost reduction, quick time to market, proliferation of computers –hence growing complexity of software systems hence growing complexity in Testing)
b. Tester’s place in overall Software Engineering eco-system (conflicting roles and responsibilities)
i. Developers
ii. Business analysts
iii. Sales and Marketing
iv. Stakeholders
v. End users
2. Project management
a. Predictability - questions like - when you will be done? How far to go? How much time it will take and how much does it cost?
b. Dependency on Business Analysts (specs) and Developers (code delivery) and Project manager expectations (deadlines)
c. Accountability issue – What if a bug is missed from testing?
d. Impossibility of 100% Testing
e. Testing resources are limited, added late in the project cycle
f. When development delays – Testing time gets chopped – Deadline remains – when the testing misses a bug – heads are rolled in testing.
3. Justification for Existence issue
a. Objective of testing
b. “Anyone can do” Notion
c. Testing as quality Gatekeeper
d. Awareness issue – how make others especially stakeholders understand value of testing.
e. Outsourcing
4. Hiring and Managing testers (Performance monitoring)
a. Skill issue: what is important skill to look for in a good tester – Technical knowledge, business domain knowledge, Test process knowledge (all folklore and legacy), Formal Techniques and methodologies in testing, Good learning and thinking capabilities and so …
b. Objective goals – How do you know tester has done a good job
c. Measuring the performance of testers by # of bugs logged
d. Notion of “Tester needs to have deep technical knowledge (not mere process stuff) so that he can get respect from development
e. Notion of “Testers needs to be a business domain expert “– Modern day interview question – “What all domains you have worked on “? Do you have any experience on “Telecom (that too billing) domain, health care, Do you have knowledge in Capital Markets? And so on…
f. Notion of “We need testers who can code”
5. Tools and mechanical Part of testing
a. Automated Testing (not Automated Test execution)
b. Regression Testing ( Repeatability Argument)
c. Testing is a branch of Computer science (yes or no?)
6. Philosophy of Testing Issue – what do you think a tester should do?
a. Handling Process and Metrics fanatics
b. Testing as Factor assembly line (test cases IN, Results/metrics OUT)
c. Find bugs – more bugs – better Testing
d. Prove that software works
e. Quality Assurance/Quality
7. Lack of Education and Skill development programs
a. General Awareness among the community
b. Problems with certifications
c. Formal university Programs
d. Research
Shrini
While thinking about asking a thought provoking question to make James Bach - write about it, I got this idea. Here is a post that lists top 7 (want to reach 10) burning issues in software testing today. I solicit other issues and thoughts about how to address them … Write to me, I will consolidate them post back on this blog …
1. External Issues
a. Business pressure (cost reduction, quick time to market, proliferation of computers –hence growing complexity of software systems hence growing complexity in Testing)
b. Tester’s place in overall Software Engineering eco-system (conflicting roles and responsibilities)
i. Developers
ii. Business analysts
iii. Sales and Marketing
iv. Stakeholders
v. End users
2. Project management
a. Predictability - questions like - when you will be done? How far to go? How much time it will take and how much does it cost?
b. Dependency on Business Analysts (specs) and Developers (code delivery) and Project manager expectations (deadlines)
c. Accountability issue – What if a bug is missed from testing?
d. Impossibility of 100% Testing
e. Testing resources are limited, added late in the project cycle
f. When development delays – Testing time gets chopped – Deadline remains – when the testing misses a bug – heads are rolled in testing.
3. Justification for Existence issue
a. Objective of testing
b. “Anyone can do” Notion
c. Testing as quality Gatekeeper
d. Awareness issue – how make others especially stakeholders understand value of testing.
e. Outsourcing
4. Hiring and Managing testers (Performance monitoring)
a. Skill issue: what is important skill to look for in a good tester – Technical knowledge, business domain knowledge, Test process knowledge (all folklore and legacy), Formal Techniques and methodologies in testing, Good learning and thinking capabilities and so …
b. Objective goals – How do you know tester has done a good job
c. Measuring the performance of testers by # of bugs logged
d. Notion of “Tester needs to have deep technical knowledge (not mere process stuff) so that he can get respect from development
e. Notion of “Testers needs to be a business domain expert “– Modern day interview question – “What all domains you have worked on “? Do you have any experience on “Telecom (that too billing) domain, health care, Do you have knowledge in Capital Markets? And so on…
f. Notion of “We need testers who can code”
5. Tools and mechanical Part of testing
a. Automated Testing (not Automated Test execution)
b. Regression Testing ( Repeatability Argument)
c. Testing is a branch of Computer science (yes or no?)
6. Philosophy of Testing Issue – what do you think a tester should do?
a. Handling Process and Metrics fanatics
b. Testing as Factor assembly line (test cases IN, Results/metrics OUT)
c. Find bugs – more bugs – better Testing
d. Prove that software works
e. Quality Assurance/Quality
7. Lack of Education and Skill development programs
a. General Awareness among the community
b. Problems with certifications
c. Formal university Programs
d. Research
Shrini
Thursday, September 07, 2006
Ask James Bach - a question on Testing...
James Bach has a post on his blog inviting questions on Testing. This open invitation comes with a rider ... only *interesting* questions will be answered, rest will ignored and best ones will be *awarded* with James writing a whole new blog post on it. I am still thinking on coming up with a question that will make James to write a seperate Blog post - that will be a real question.
Just to remind you the commenting policies on James' blog --- Any comments that makes to his blog post (after moderation) are considered to be useful to the readers of the blog (as endorsed by James himself).In my opinion that is a like "treat" to me when my comment makes it to comments list.
So what are you waiting for ... Just grab the opportunity ... Ask James a nice question on Testing ...
Just to remind you the commenting policies on James' blog --- Any comments that makes to his blog post (after moderation) are considered to be useful to the readers of the blog (as endorsed by James himself).In my opinion that is a like "treat" to me when my comment makes it to comments list.
So what are you waiting for ... Just grab the opportunity ... Ask James a nice question on Testing ...
Subscribe to:
Posts (Atom)