Friday, May 21, 2021

What is Test Design and Test modeling

 Test modelling - Activity of representing system/feature under test as a model. 

A sample hierarchy

System

        Entities/Objects/Users (Roles)

            Dependencies between objects

                Relationships - IS A and HAS A (Type and containment)

                    One to one, One to Many

                        Properties (variables and constants)

                                Values (domain) of properties

                                    Default values, boundaries of values

                                        Combinations of Values to create test scenarios


Object - Action - State Change.

List actions on each objects.

Create a map of what will happen when an action on object happens

Simultaneous actions

Dead locks

Hangs

how do we use  BB design techniques

EQ partitioning

State machine

Decision Tree

Combinatorics


Take  a par of variables and create a Grid.

GUI Testing check lists (applicable to new GUI)


Static vs Dynamic Models

Model - when nothing is happening. No user interaction

Dynamic - when users are interacting

Time element


It might help to user Object Oriented Design to represent above idea.

There are objects - have properties and functions.

Nouns and Verbs


System will have objects/entities. Objects have properties, properties can be constant or variable.

Take each variable - explore the domain of the variable. See what kind of values it can take and 

String these variable values into test scenarios.


This act of modeling - goes through iterations. It's like a photo film negative development. With each iteration  - various features of system start showing up.


Idea of Test coverage -

Imagine a 2 D shape - area of features in two dimensions.

Covering area means exercising the walking through the area.

how about Volume ? 3rd dimension - features and values

Wednesday, May 19, 2021

How to improve Questioning skills

 Following up on a post in The test tribe FB group - here are my toughs on improving questioning skills

First - few ground rules on questioning

  • Questioning is a skill - improvable on deliberate practice
  • Questioning does not mean being disrespectful to other party
  • At times questioning might be taken as challenging the authority - deal with it appropriately.
  • Empty your ego and false sense of "I know this stuff" - instead approach "what else I don't' know here" What else is hidden from me now.
  • Frame questions and follow-ups in a way that it does not turn off or irritate other party
  • One good way to start question is to describe or re-phrase the understanding in words of other person wherever possible and seek view on if your understanding is correct
  • No question is silly (from the view point of the asker...:) Only unasked question is silly
  • while answer is being given - listen carefully - where applicable watch body language of the responded. Do not rush or pounce on  immediately.
  • Listen with an intention to understand not with an intention to answer in return
  • If you question basic ideas - it will not make you less in any way or a fool
  • Display courage and humility

Tips/Practice ideas
  • Ask one question at a time (Courtesy : Jerry Weinberg narrated by Ajay Balamurugadsa) 
  • Approach everything with "awe" and a sense of wonder.
  • Approach everything with mind of "newness or not seen before"
  • Develop good vocabulary and skill to say one thing in many different ways
  • Be good with creating examples and analogies
  • Dramatize your answers and questions
  • Do a role play
  • Create diagrams 
  • Ask "What does *this* mean? 
  • Seek meanings , interpretations and context of words
  • Learn about word etymologies
  • Develop understanding through responses received
  • Do lots of imaginations
  • Show child like curiosity
  • Split words from sentences - ask "what does this mean" to every word
  • ask "what id"
  • Ask what are constrains or limitations here - what if these constrains are not there - for example - what if we can fly defying the gravity or what it we can walk on water
  • Question universally known and understood words and ideas
  • Question authority
  • Do not settle for right answer - seek more broader set of responses
  • Be respectful
Human relationship aspects of Questioning
  • You always run into risk being perceived as arrogant, egoistic, "know all" OR completely dumb
  • You are mostly likely to put people out of their comfort zone and make them feel nervous
  • People may not speak to you nicely or avoid talking to you
  • Understand emotional aspects

Types of Questions:
  • Open Ended -  Non conclusive answers
  • Specific - 
  • Binary - Yes/No
  • Questions to check knowledge
  • Questions to understand

Tuesday, November 24, 2020

What's big deal with testers? Hit that punching bag !!!!

 Once in every 1 or 2 years - testing team or function comes under "question". I have seen this consistently for last 20 years. For business leaders, Heads of Business verticals - testing has been a punching bag. Whenever leadership changes or sales go low or profits fall - suddenly discussion on "efficiency" and throughput per person becomes an important metric. Why do we need such a big testing team? why cannot developers do all the testing required? Why not end users do the testing required? why cannot be automate this stuff and get testing head count out of equation? Why cannot we do crowd sourcing? - are the questions that business leaders ask. Testing leads and managers - start running from pillar to post to justify why they exist -- come up with all sorts weird metrics and process maps to show why testing required. But business leaders are hell bent on cutting that "extra" fat in the system. There go "testing team " out of the company.

What is big deal about testing or testers ? They ask. First of all let us ask what is/ required to do testing? Knowledge of the system or business domain or technology that used to build it?  Let us explore...

Business analysts know the product inside out as they have "defined" it (by writing down BRD - business requirements document). They should be the ones best suited to do all or required testing. No?

Developers too claim that they know system inside out  - as they have created it. They know nuts and bolts of the system where each one is. Should they be the best ones to test the system?

End users are best to test the system as they are the ones that eventually use the system they know what should be there in the application. Are the the best party to test?

Before we respond to above possibilities - let us also look at how industries shaped the views about testing. In software world (or software enabled businesses), broadly there are 3 types of industries - Software product companies, IT services companies and IT organizations or captive units of other (non software) businesses - likes of banks or pharma or Automobiles etc. Each of these have a very specific and unique "culture" about software and hence about testing.

Software product companies call software development as "Engineering" and all activities about creating a software is put under one umbrella "software engineering". Testing is part of "engineering". Pre-agile Era had testing as part of "development" life cycle and developers did all testing that they could. Agile and Post Agile (now DevOps era) killed "testing" function or skill or requirement by few beliefs or practices. Agilists extend logic of all testing should be done by developers and said "quality" is every ones' responsibility. Thru practices like TDD and formalized unit testing - testing as seen as excercising or executing every line of code (they wrote). Testing would mean writing code - hence programming and testing at some point merged into one skill. If you are in an engineering team you did not do coding means - you are either a business analyst or project manager (or scrum master - not ring master !!! :) 

IT organizations  or captive units are next big communities - looked at testing as either some necessary evil or something that programmers or technologists should do. Business leaders of these groups saw entire software development as an alien thing and kept a safe distance from it. For these software or technology was a support function and thought any investment in activities (like testing) related to software development as "distraction" from their main business function. Technology teams in such organizations too had a strange problem or approach. They thought all required knowledge of business domain or application resides with business so technology job is sincerely translating whatever business gave as requirements. No questions asked. They also propagated a view that technology testing is a shallow happy path and basic validation. They believed its not their cup of tea to learn business domain.  So where did  bulk of testing go? To business and End users in the form of "User acceptance Testing".

IT services companies did not have any of such challenges about thinking about domain, business, technology and testing. They simply supplied what their clients wanted. Hence IT services companies had mix of both IT org and product company cultures. IT services companies did not have to have their testing culture. In the beginning (pre-2000) - by riding outsourcing wave - went and told their outsourcing bosses - "testing is low risk work - we can take it away and do it for very low cost". I have heard pricing models of testing services on unit basis - xx $$ per test case. How crazy was that? When they ran out of business supplying "low skilled" (almost brain dead) "executors" of test cases (called manual testers) - they started selling them as business analysts. In that parlance - business analysis did some testing but knew business domain and could "write" BRD's. Even today many testers think that "business analysts" are one notch above testers in terms of salary and org hierarchy.  That option too soon ran out. These companies then started selling failed or low skilled developers as "automation engineers". This going even today..... skilled testing has been dying... not dead yet.

I ask -- what is big deal about testing - what makes testing  "tick" in todays situation? Share your views.



 


Sunday, November 08, 2020

My journey of Software Testing - Looking 20 years back - Part 1

“You seemed to have a good eye and aptitude about find bugs in software. Why don’t you pursue a career in Testing/QA? “said Premjit my manager at i2 Technologies roughly about 20 years ago. As it turns out he was damn right in assessing my special skill and interest for looking for bugs in software. At that time – like many other peers I felt Premjit was trying to put me down by asking me pick up a career in Testing/QA while I was aspiring to be a software developer and create awesome applications. That time (true in most of jobs even now), developer meant to write/design code while tester means to “test” (use) those applications to find bugs. Honestly, I felt, it was a let-down for me.

I reluctantly took full time testing role (at that time, in software product companies, testing was a part time job and everyone were called as software engineers). I have to admit that it was decision born out of fear of losing job. But in retrospect – here I am 20 years later – celebrating that shift Premjit suggested me. Post that shift, initial few years were bumpy, I kept looking at my developer friends and used to envy them. In initial few years I took roles that are closer to development while my full-time role was to test. I was software librarian for a new product development - teaching and establishing software versioning and build management. It was interesting as I could hear few developers appreciating my knowledge development practices, IDE (Visual studio) know-how etc. Soon, testing component of my job increased and I became moulded well into doing testing. I went on investing in learning about testing deeper – I took a course (2-day workshop) by T Ashok of Stag Software – who happens to be my first guru on software testing. He still continues to be a great friend, guide and mentor. I attended a testing conference by STeP -in in 2004 – got convinced that the career I took up indeed has good prospect – looking at and hearing all those leaders from industry who were managing testing teams. I was introduced to leaders like Srinivasan Desikan – I still continues to be in touch with him. In another interesting encounter in same year, in QAI conference, I meet Vipul Kochar – good friend with whom I continue to share good collaboration on software testing related initiatives in India. Vipul in his talk (in that QAI conference) on exploratory testing – introduces me to the works of James Bach who still continues to shape up my thinking and influences me about “fearless and think—for--yourself” approach to testing.  That’s I think happens to be a major turning point in my career in testing. That QAI conference was a unique experience as that was my first instance of speaking in a conference, guess what!!!  topic of my talk was “Agile Testing – is history repeating itself”.  I also meet Krishna Rajan in that conference who would 6 years later would give a job for me at Barclays. So, in many ways 2004 was an eventful year for me.

In the mean time I made transition in to IT services world and started learning more about testing by reading through works of James Bach and Cem Kaner. I then came to know about Context Driven Testing school of testing. There used be a yahoo user group called “software testing” where James, Cem and others from CDT were regular contributors.

I vividly recall a post (query – actually) in that group where I expressed my doubts of testing as I saw during that period – 2005. This interaction on that forum with James Bach as a clear eye opener for me and would set me in quest of knowing and learning more about testing in coming years.

Here, I share few snippets of my questions and answers by James. Read carefully. This is how a newbie tester starts off.

My first question about testing was about role of Domain expertise. Major view at that time around me (it is even now largely persisting) was for doing good testing and excel in that one needs to be a domain expert.

I ask “In testing domain expertise is important - for example without knowledge of stock markets – you cannot test an application that is meant for stock markets. I agree with this notion but banking only on this is recipe for failure.”

James responded to this – “Someone without specialized domain expertise can contribute to a test
project within any domain. Besides, some domains aren't that challenging to pick up, and in some cases, the test oracles aren't that difficult.  However, trying to test, say, a radiology imaging system using only testers who have no knowledge of radiology would seem dangerous to me.”

That as an important lesson to me – while having domain knowledge in area application is a good thing but that is not a mandatory thing. You can contribute to testing by various other skills in spite of being a newbie in that business domain.

My next question was [paraphrase] “Any developer can do testing (he/she does any way unit testing and other forms of developer testing). So, if we train him/her on automation tools - we are ready for state-of-the-art Testing. Why we need a separate role for testing?”

James responded – “Any developer can do testing. So can any plumber, any housewife, or any politician. Anyone at all can do testing. Do they do it well? That's the question. Developers bring certain skills to testing that I like to have in my test group. They also bring certain biases.”

This taught me why it is important just not do testing but doing it well – like a specialist would do. Anyone can do testing – that is a fundamental human trait to check things. But a professional tester would do testing “well”.  Developer Testing has its own biases. I was introduced to the idea of what developers often miss when they test – conformance oriented or confirmation heavy testing.  When you are a professional tester – you would look at the task of testing as if your whole world depends upon how well you do it. It’s no longer a ‘task” that had to be done. This thought made a huge impact on my thought process of testing.

I, then asked about another popular field of QA Process” – Why certain companies bank on quality jargons - CMM, ISO, Six Sigma saying that testing and QA are one and the same. Further they claim they follow CMM /ISO/Six sigma based so quality is obvious outcome – we don’t consider testing as specialized skill. We are fine with our developers doing all required testing.

James dismissed this by saying “Well, that's just corporate religion. I'm frustrated with that, too.”

This was a burning question that I had at that time “As mentioned in points 1, 2 above, if we have developer who can do  testing  with automation tools and business experts who bring  the necessary business knowledge and speed/efficiency in testing - where is the scope for  Tester - How do I emphasise/sell  the concept of " testing" being a unique software life  cycle activity and brings definite value to end product. In other words, in what way a tester is different (in skill sets and nature of work) than developers and business consultants or analysts?  What is the need for hiring testers instead of developers doing testing too?”

James responds – “I think if you don't have a clear idea, yourself, about what testers do and what skills they have, you won't be able to convince anyone else. Here's one skill: the skill of making models. This is a sub-skill, in my reckoning, of general systems thinking. Another skill is critical thinking, which involves using logic, of course, including the process of abductive inference. These skills are difficult to evaluate in an interview. Hiring people who aren't developers is worthwhile for a few reasons: there's more people to choose from, they probably have different biases, they won't be trying to get into a programming job, they will probably be better at testing overall and they will become better testers over time. I like hiring people with a philosophy education, when I can find them. You have to *demonstrate* the value of testing culture, in order to sell testing culture.  Read the articles and class materials on my website. Go to www.testingeducation.org and take the BBST online testing class.

I learnt through this response from James that – I did not have clear idea of what good testing was myself. I was just observing industry trends, what I hear in conferences and what my peers speak about testing. I was also constantly distracted by “jobs” that are related to testing but not testing actually.

Looking back, I laugh at myself and raw thinking. But these interactions would lay good foundation for my future learning.  My continued asking about my doubts about testing with likes of James and others Context Driven testing community in coming years would make be a better tester and would put me in a position that I can answer questions like these to myself.

To be continued ….

 


Monday, June 24, 2019

There is no such thing called defect/bug in Machine Learning/AI domain

One question that comes up again and again in Testing world today is about role of testing in the domain of applications in Machine learning and Artificial Intelligence. To be precise, many in testing community are curious and some-what confused about what they need to do differently (if at all) and what skills they need to acquire additionally. This is post is an initial attempt to share my thoughts in this direction.

What is an ML Application ?
(Machine Learning is considered to be a branch of Artificial Intelligence, hence Omitting using AI along with ML)
The term "Machine Learning" is not new, it was coined by Arthur Samuel in 1950. Definition given by Arthur was "ability of computers to learn without being explicitly being programmed".  In reality, computers do not learn, but software programs learn - a small difference, if you chose to care. How do programs gain such ability to demonstrate such human-like ability to learn? Any any or every program be made to "learn" like this? What has enabled today's computer's technology enabled such possibility being realized? Answers to these questions take the post beyond the topic about ML, Testing and defects/bugs. In short - I would say ability of computers to store and process large volumes of data at the speed needed at processing transactions - has enabled Machine learning as Arthur Samuel might have envisaged.

What is Machine Learning application then? A program that  uses a set of algorithms processing sets of specially selected and curated data about a problem that program intends to solve. Under the hood, the algorithms "fit" the data to some selected mathematical "function" called as "model" such that the programs logic is data driven not hard coded. When I say hard coded in ML parlance - you will not find explicit chunks of if-else or select-case or do-while depicting rules of logic. The "model" through "fitting", generates the logic that data presented to it shall comply.

What kind of problems ML programs can solve? Largely two categories of problems - prediction and suggestion. A machine learning program can classify a bunch of financial transactions (say credit card) as fraudulent (potentially) or genuine or recognize faces in a picture or auto complete what you are typing in a search box on a web page. 

What does it mean for a program to learn ?
In simple language - learning for a program is to discover parameters of mathematical function that program uses to establish relation between input and output. Let us take an example of classification that aims to predict whether an image contains text or not. In this case the image and its properties (what each pixel tells about the whole picture) are inputs and output is a binary decision whether image contains text or not (1 or 0). For a human eye - it is easy to make the decision where as for computer  - the problem needs to be presented as (an example) a mathematical function like y =f (x). This function will have its parameters that the program needs to compute. For this purpose the program needs to presented with loads of data (input images and decision whether there is text is there or not). By processing this data the program is expected to identify the relation between "y" and "x" which is a mathematical function like y=mx+c (here m and c are parameters of the function).
This process of arriving at parameters of the function by working through data is called as "learning". Once the program learns the relationship - then, it can predict "y" - decision that whether image contains text of not - given any new image that program has not "seen" before.

Needless to say computer (program) does not "see" the image like a human eye - it (program) sees the image as a matrix of numbers that indicate pixel color scale or density. There easy python modules/programs that can convert an image into a matrix of numbers that a learning program can consume.

Also important to note all that data that the program has "seen" or processed during the process of "learning" does not stay with the program. What is left in the program is just the "essence" of data that leads to establishing the relationship y=f(x) in the form of parameters of the function. The data that program uses to "learn" the relationship is called as "Training Data" - how innovative !!!

Coming back to main topic of the post - what does a bug mean in this context ? When a program incorrectly calls an image as containing text when image does not contain text  - do we call that behavior as application bug? ML programmer would probably call  it as "program is learning" or "program needs to see more data to increase its accuracy of prediction". In this way - every opportunity for program is learning, like we say a lawyer or doctor as "practicing" - ML program, probably never "performs" but always in the process of "learning" !!!

What do you say? If program does learning (I have dislike for the term "machine learning" as its not machine that learning - its the program that is learning. Try saying programming learning, or software learning !!! its funny) - what testers need to learn ? What is left for testers to learn if programs become intelligent ?

Wednesday, May 15, 2019

Industrialisation of Testing, Heuristics and Mindfulness

Over last two week end - The Test Tribe (popular testing community) hosted two sessions on facebook - one from T Ashok on Smart QA and other from James Bach on "Testing Heuristics". Both sessions were well received and interestingly I could see some connection between ideas that were part of these two sessions.

Industrialisation of Testing - Up until now - I thought industrialization in testing as bring "factory" metaphor into what we do as testers - intellectual search for problems in products we test. Ashok T in his session took a different position. He says industrialization in testing is about doing less through exploiting work done by fellow testers in the form of tools, test ideas, methods etc. He drew parallel with how software development community though its open source revolution - makes it possible to build application with writing less and less code. He stressed on creating open source revolution in testing so that testers can share their ideas so that we can use, reuse and grow testing repository. That would be true industrialization. There has been such work happening in our community - what we need a platform and such active participation/contribution.

Mindfulness Ashok in his session urged testers on mindfulness - acting with awareness of how we work, why we do what we do. Very nature of the mind is such that it wants wander and then programs in subconscious mind take over - run the what we do without our conscious engagement. Testers through their habits go about their day's business without being consciously aware of decisions, choices they make. Through mindfulness, testers would need to break the autopilot mode and carefully watch every step - this will enhance their skill, productivity and reduce errors they make in their work. Rarely I have seen such an advise to testers  - indeed a point to note.

Heuristics  James Bach in his session on Heuristics - went on in detail to explain how all testing, software development and Engineering is rooted in heuristics - fallible methods to solve problems. Those who follow context driven testing community are well aware of this term. James explained how heuristics need human judgement not mere following the rule -as heuristics can fail. James said in our daily life we use many heuristics without being aware. He urges, from his own training and experience, to be aware and name a heuristic when you use one.

Here is where I am reminded of mindfulness that Ashok suggested to use. By being mindful -we can recognize heuristics we use, when we recognize , we can name them, when we name them - we can share with fellow testers. That leads a community movement which manifests as Testing industrialization. Its exciting to see these two testing guru's ideas are connected in unimaginable ways. 

Sunday, September 16, 2018

Testers don't and can't prevent bugs : Alltruism or Sense of Pride ?

One of the fashion statement associated with testing these days is "testers should focus on preventing bugs rather than finding them". This is a very tricky idea and is full of traps for testers. Recently a post came up in software testing yahoo group that somehow got into this topic of preventing bugs.

Coming from the context driven school of testing and trained by likes of James Bach, Cem Kaner, Michael Bolton and others - I was skeptical about testers preventing bugs. Fundamental idea of our school of testing has been that as testers we bring to the light the information about bugs and risks in the software we test. Then we report it in a way to stakeholders (powers to be) to act on it.

Many testers fall into trap and take upon themselves (may be due to role/corporate hierarchy pressure) to task of preventing bugs. After all - who does not like someone who prevents bugs than someone who simply reports. Borrowing from manufacturing industry - many business leaders in IT and IT enabled business - firmly believe in prevention is better than cure. Who can resist the nobleness of preventing or saving "nine" by stitching in time.

Let us consider following two cases -

Testers prevent bugs in the requirements by asking question about ambiguity in requirements. Requirement bugs might not be counted as bugs by many - they might be termed as unclear requirements. Calling out what is not clear in requirements is one of valuable contribution of testers.

When Pairing with developers - testers prevent bugs as and when bugs are occur. For example tester may shout .. "hey you are missing exception handling code for that exception or hey you got if loop condition incorrect". That is closest you can get in preventing bugs.

In an email conversation with Michael says -

"we do not use a binary model “pass or fail”.  People who do that are setting themselves up for bad testing.  A product—any product—can “pass” a test but still have terrible problems.  A product can “fail” a test, yet  there’s no problem.  (For instance:  the square root of 2 is 1.4142136, right?  Well, it isn’t; the square root of two is not a rational number; it never ends, and certainly lot at the seventh decimal place.  But for many—even most—circumstances, 1.4142136 is good enough; just fine; not a problem."

This has been a great learning -- testers throw light on ambiguity - that does not mean they prevent bugs happening. Similarly,  in pair testing - testers spot the bug in shortest time possible but they did not present it from happening... that is "early bug detection".

Thanks James and Michael - lesson re-affirmed.

Sunday, March 04, 2018

Chief Value Officer vs. Chief Feelings Officer - Perills of Reification !!!

"Yet the danger if reification is all too real. We fall in love with our models, yet we need to be reminded that they are just models of the real world." Lynn Chiu


A good friend of mine Ray Arell in a tweet asks "why not have a CVO - Chief Value Officer". The word "value" always evoked a very strong internal response in me when I saw it being used in a way that Ray used. This word like others in the same league such as "Quality", "Customer Experience" - is notorious or victim of being reified (not rectified). Michael Bolton first introduced me to this word when we were discussing about abstract vs. concrete things. I learned from Michael that reification leads to gross misrepresentation of idea/word and leads to "gamification". It is a thinking fallacy and all intellectuals/thinkers need to be alert about such thing happening.

What is Reification?
In simple terms - reification refers to considering an abstract idea as though it is a concrete, countable, measurable thing. It is about wrongly understanding an idea as a thing. For example counting how many ideas are generated in a brain storming session is an act of reification as idea is not a thing - counting and doing all sorts of maths around it does not make sense. Here we say "idea" is reified as a thing. Other examples include making objects out of subject human experiences like emotions, feelings, values (say family values, social values) etc.

Marxist definition of reification is about "thingification" of social relationships. Among several perspectives and meanings for this term - I would like to use this definition for the purpose of this post - "A fallacy of treating abstraction as though it is a real thing"

Why reification is problematic?
First of all - reification is misinterpretation of reality of nature of what we are dealing with. It's a fallacy, an error in thinking and communicating. Reifying an idea into object is to strip off the subjectivity, mystery and complex richness of the idea. One common outcome of reified communication is both giver and reciever will have two different meanings and interpretation of what is being conveyed.

Consider yet another term "Quality". In software testing world we are all are familiar with this word. There are more than dozens of definitions of this word each fitting to a specific context and it demonstrates how the term quality offers itself to reification. Quality stands a mask for so many desirable attributes of a thing or a service. Instead of adjectives like "fast", robust, flexible, Easy to understand, cheap we can say quality and get away with bothering about all the specificity and correctness of what we want actually. That is power of reification but that is incorrect, manipulative and bad way to communicate.

Similarly with respect to motivation and change management - we often commit reification error. Social constructs such "percentage of work completion" is often regarded as measurements of real objects when it is the best an idea.

In the word of testing - there are famous examples of victims of reification. Requirements, Test cases and bugs. All these are complex ideas being generated as part of our quest to create software from requirements specified in natural language that gets interpreted and implemented into formal computer language. In the word of agile, we have stories that now replace requirements. A development lead announces in the first sprint meeting of a project  "we plan to delivery 18 stories in this sprint".  18 what ? stories. A test lead is asked "how many test cases you team plans to execute in this release"? In another case, during a project postmortem meeting - a comparison is made between number bugs logged in a given release to the number corresponding to previous release to assess the quality of "this" release. 18 stories, 3000 test cases, 270 bugs are examples of how in today's software world we ruthlessly reify abstract ideas and do math with these numbers. The act of reification allows use the numbers that do not have any inherent meaning of their own when context, giver, recipient and time are removed from them. What happens there after is pure game of manipulation.

Reification is thinking error ... it is a fallacy.

Value vs. Feelings 
In today's business world - the word "Value" is more attractive and sexy. We have terms like value stream, value proposition, value added service etc. Behind each of these phrases, hides a very clear objective, object or a concrete thing. It might be some money, timeline commitment, specific characteristic or outcome of a goods or service. It has become fashionable to use the term "value" instead. Why ? Since the meaning of value is subjective and open for interpretation - it allows one to use word value and imply one thing and later for the same value imply something else. In a sense - using value allows one to manipulate the situation to his/her advantage while not being wrong or incorrect about what is being conveyed through this loaded word "value".

In order to understand the full and correct meaning of word value - we require the context and who are we addressing to. By reifying the word value - we strip off that that richness, context and complexity. Then we start using the phrase to indicate multiple, sometimes disconnected and contradicting ideas as we have left behind context and recipient(s).


Instead of having a Chief Value officer, let us have a Chief Feelings Officer who can understand and deal with customer feelings and emotions about a goods or service delivery. Having this role, corporates can truly claim that they care about individual views and feelings about customers than a rolled, convoluted, metricized - measure such as customer experience.

Many still think that giving good customer experience is having a great looking GUI and exciting animation. Real customer experience, in my opinion is about caring for individual experience in their bare essential with all richness of emotion and context.

CFO - Chief Feelings Officer. Anyone ?

Saturday, November 18, 2017

Computer does what programmer asks it do : why there are bugs?

A colleague of mine said something so extraordinary about software bugs that I have never seen anyone talking about software bugs that way.  The discuss was about how current technologies and advances in Big Data, Machine learning and AI have or will change the way we do testing and how these can help testers in testing.  One of the underlying applications of these technologies is two fold approach - one mimic human action (vision, speech, hearing and thinking !!!!) and then make predictions about what will happen next.

When it comes prediction and testing, obvious topic is "defect/bug prediction".  Bugs are hardest things to predict due their very definition and nature.  This colleague of mine said something that captures this sentiment very well - "There are no bugs in a sense that computer (he wanted to say software... these days it has become a fashion to replace the word software to machine at all possible instances) does not malfunction on its own (barring hardware/power failures etc). Computer does what programmer wants it to do or coded it to do. The problem then lies with human programmer's mind (or brain) that gave computer an incorrect instruction."

Where does this takes us to? It follows from my colleague's logic that the problem then lies with programmer's mind that gave computer the "wrong" instruction. Predicting a bug then would mean predicting when a programmer gives wrong instruction. This is a hopeless pursuit as guessing when human mistake is unsolvable puzzle - at the most you have some heuristics.

If we go back to the idea that software bug occurs when programmer gives a wrong instruction to computer. This line of investigation is remarkable -- First of all how to identify an wrong instruction?
It turns out that a wrong instruction cannot be identified using say an algorithm or mathematical approach. An instruction (such as open a file, send a message to an inbox, save a picture) becomes "wrong" not by itself but the context or logic or user need or requirement. This then takes us straight to mechanism using which we specify the context, need or logic. That is the realm of "natural language".

Software bugs happen due to programmer "wrongly" translating a requirement which is in natural language to a world of computer language.  If we were to predict bugs using likes of Machine learning or AI - we need tools to spot this incorrect translation.

Looks promising ... right? The state of the art in Natural Language Processing (NLP) is about how closely computers (software actually....) can understand natural language. There are  stunning applications of NLP already.

When NLP comes close to understanding human language fullest - we move a step forward in the puzzle of spotting incorrect translation of software requirement to a computer instruction. I hope so....

But then nature (human) leaps to next puzzle for computers... limit of human intelligence and vastness of human communication. With brightest of human testers, we often fail to spot bugs in software - how an approximate and "artificial" system that mimics a portion of human capability do better in spotting bugs? An area to ponder .....
BTW - was my colleague right in saying "computer exactly does what programmer has asked it to do" Really ?


Thursday, August 10, 2017

Machine learning and Software testing

Machines are learning - good for them. What about humans? Popular buzz around now is about machine learning and artificial intelligence. Never in the past, I think these terms intelligence and learning - have become so much importance and got prime time media coverage than now. Thanks, ironically to the qualifiers attached to these words - Artificial and Machine. Now days more engineers are investing time in learning how machines learn (what a paradox) and intelligence that is fake... sorry artificial gets more funding and attention. Has value and quality of human intelligence gone down or has human learning stopped ?

One of the common and popular use case or illustration of machine learning is that now a machine (a software program actually) can recognize picture of a cat or an apple, several types of apples and cats without being explicitly coded do that. Whats more ? As this program "sees"more and more apples and cats - it "learns" - gets better at accuracy at identifying objects. That's quick machine learning intro for you.

When someone takes this idea of identification of car/apple by machine and asks "why cannot machine identify a software bug - as this person does in introduction of this video (at 1:09) - a paradigm shift is needed.

Let us face it - what are in common between a program identifying a cat or an apple on the screen to some other program identifying a bug in a software ?

1. A program with its code and machine learning capability- does its job with relatively simple and formally defined model. There would be rules and patterns in the model to assist the identification. Where as when it comes to form, shape and identification marks for a software bug - you will really struggle to define it.A machine learning model that can recognize a software bug needs far deeper and complicated definition of bug.

2. Even if you concede - you have managed to define a model that can recognize a software bug, the real challenge would be identifying it in a real time when software is running.

Identifying a software bug in simple sense would need following
- Mechanism to generate loads of inputs and configurations of systems under test
- Mechanism to operate SUT with these data sets and observe potentially large number of possible software behaviors
- Among possible outcomes - identify the buggy behavior (Oracle problem)

In short - these are hard problems of software testing in the first place. How machine learning can help?

I like what Paul Merrill says at the end of this talk on youtube talk - "Machines are learning. Are we"(testers) ?



Hard Problems in Software Testing (2017) - Part 1

When I set to write the post with this title - I thought it must be first of its kind. It turns out there is a book written on this subject. The authors of the book list down a number of problems of testing and solution in the approach called "Testing as Service". In this post, I approach this topic from a totally different starting point.

Let me reflect on history of computing a bit to set context to software, software testing and the topic of hard problems.  The word computing refers to use of computers to solve or create systems to solve a range of problems in the areas of math, information science and like. Named after 9th century Persian mathematician, Al-Khwarizmi, the term algorithm gives a formal structure to problem solving approach. A step by step procedure or method to solve a problem is referred to as "algorithm". The program (or software) implements an algorithm and solves the problem. The algorithms can be represented in multiple ways through natural language, pseudo-code, programming languages, flow charts and control table etc.

In early 60's and 70's when computers developed as advanced calculators, math and logic enthusiasts pounced on these new creations to see if their long pending problems be solved. Few wanted to solve the problem of finding out if a given number if prime or not while others wanted to solve a shorted route for a traveling salesman. In these implementations - the program would run (in isolation - no network or internet in those days and no auto updates of OS or any other software) with an input set data set and would compute the "Answer" or "Solution".

Modern business software at the core level is built from the algorithms performing computation/information processing. In word processors, web browsers, camera app on mobile phones - you will see a culmination of work of several algorithms working in background. These algorithms solved basic problems like storing, sorting, classifying information.

Another thing that set the computational problems of 70's to that of business software of 90's and early 2000's is - introduction of Natural language (Likes of English) for specifications. The problems that algorithms solved in 70's were represented in formal mathematical notation. With the introduction of Natural language at one end and high level programming languages like COBOL, Fortran, Pascal, C, C++, Java - we created this problem of translating what is specified natural language to computer language. This created a division between those understand business domain (Natural Language) and those understand computer language (Programmers). This is first big problem of software development. By natural consequence, validating that the program did as per what is specified in natural language - also got complicated. Software Testing that branched off from software programming as a distinct activity from early 90's - has been trying to bridge the gap between programmers and business folks.

The field of computer science deals with solving computing problems and algorithms. The hard problems in algorithm world are classified as P or NP problem. Interestingly this classification is based on evaluating if the algorithm produces result (halts as in halting problem) in a polynomial time function of size of the input or not. Those problems where algorithm fails to halt or produce results in a polynomial times are referred as NP problems - Non deterministic Polynomial problems.

Where does software testing stand in this classification of P and NP problems? If an algorithm were to test a computer program - would it halt and produce answer in polynomial time? How would an algorithm approach the problem of testing software ?

Here is an attempt to list down the problems that characterize software testing as NP problem.


Each problem listed here shows an aspect of testing that makes it hard to have have an efficient, less error prone and cost effective solution. These problems are hard as solutions that we see in practice are sub-optimal and need constant refinement.

1. Problem of potentially infinite sets of Inputs
Unlike programs/algorithms of 70's - modern business software receives and processes a large set of variables and equal or more numbers of input values directly sent to the program. Also modern software is not an isolated desktop software running on one computer - but a combination of several stand alone components running on different computers connected together in a network. A software under test by virtue of this arrangement continues to receive multiple implicit inputs that influence outputs the software produces. Then we have the database/sets of data elements that are managed by the software - state of this database also influences the outcomes of software. There are internal (to the software) configurations that  allow software to be configured in many different ways.

The task of generating all or some "important" sets of direct inputs that are fed to the software while running and sets of all indirect inputs (database, network, internal product configs) - is one of the hard problem. 


2. Problem of operating the software (and its dependencies) under test through set of inputs
The largest chunk of time of testing is spent in operating the software once we have configured software under test and its dependencies. A simple and single thread of this "operation" is the part of a larger unit called as "test case" or "test" that additionally involves making observations and inferences about outcomes of the "tests". Given infinitely large number of inputs (direct and indirect) there are equal number of ways of operating the SUT. This is hard problem. How can we run these "tests" in a finite time and resources? Who would run these tests? Human tester?

Then we will have questions about how these tests be specified, in what language and how detailed. We have attempted to use in both natural language (manual test case/script) and software language (Junit class). How to run these tests - we have tried "interfaces" of the SUT for this purpose. Most popular interface - GUI created an industry of test automation tools and the paradigm of "record" and playback". Some geeky programmers used interfaces like web service to execute the tests in an non interactive way. Both of these approaches have met success to a degree but have left lot to be desired.

The task of running tests - operating the software through a large set of inputs/flows is a hard problem that we need to solve, solve well.

3. The problem of Observing direct and indirect outcomes/behaviors
While programs of 70's produced one or more distinct outcomes as solution for a given problem - we in today's world need to world need to observe software behaviors. It is funny that we use term "behavior" to inanimate object like "software".

Like direct and indirect inputs that the software takes while in operation - an important puzzle of software testing is about observing "all possible" outcomes. How do we do that? Again - there is a human way and an automated way. Continuing on the testing task of running tests - you might argue that making observations on outcomes is extension of executing tests. This is true by and large. The challenge is to specify what all to observe and how. An automated test  might say watch this space or this folder or look for this text message and so on. But that is only part of the test. Given a test, SUT shows many different behaviors and Capturing all of them is a hard problem. More than that - how do we know we have in our list all that we need to observe?

4. The problem of identifying correct and incorrect behaviors - problem of test oracles

On the contrary to what we believe, it is often not very clear as which software outcome is correct which one is a bug. To help in deciding, we use a reference or mechanism that can decide the correct behavior. Requirements specifications give first reference to what we should expect from software - in natural language. Given infinite sets of inputs and corresponding outcomes and behaviors - identifying the right and correct behavior requires a very large number of oracles.

More often than not, humans can and do act at live oracles - they use their own experience and some given references can identify correct behaviors. At times - data and captured behaviors or previous versions (assumed to be correct) of the application is used as test oracle.

5. Biggest of all - repeating all above many times, when software changes
Software is soft and when it is changed, many things change that are not expected to be changed. This is referred as regression. In the life of software, several times it needs to be changed, updated and new features and capabilities to be included - when such change happens, it is not enough to test and validate the changed areas/features - often we need to confirm that changes made did not break other working parts of the software. This means a continued effort and work testing software completely (almost) at all times when there is a change. To make matters worse, you need to do so called "regression testing" even when any external software (external to SUT) is changed. This is biggest problem we need to solve in testing - the burden continuous testing of entire application and its dependencies.

6. Problem of defining and quantifying value of Testing
Testing has no direct value for customer of end user who is interested in how and what features the product offers. Customer assumes that the delivered features work as expected. The value testing in the performance of the product in the hands of the customer is roped into the larger work by the team - mainly development team. The indirect nature of contribution of testing to overall product makes it hard for testing to assert itself and ask for due share in the success/failure of the product.

Our field is about half centuries old now. How would we approach these problems of testing software if we were to start all over today?

To be continued .... in part 2

  • Problem of quantification how much testing needs to be done and how much is done
  • Problem of estimation of testing required to be done given a scope
  • Problem of Skill/ mindset
  • Problem of expectations from Testing

Thursday, August 03, 2017

Testing Maturity - Dealing with grown up Kid

Several years ago, during my days as Software testing consultant (not a doer but a consultant) – one idea that repeatedly came up was “Testing Maturity”. Thanks likes of CMM, CMMI, TMM, TMMI, Six Sigma, TQM and others – IT world was (mostly “is” as well) obsessed with knowing what it is means to be a “mature” about just anything. Testing – being one of the most talked about maturity target.

I still remember of my first experience of with testing maturity models – when searched on internet, I did not find much “state of the art” stuff (about 10-12 years back). Then like many others – I set out to create my own “framework” for assessing testing maturity. Looking back – I see my attempt as very “immature”. It pretty much looked like any other similar framework, it had levels of maturity, key focus areas and some kind of recipes to move from level 1 to level x and so on. My bosses then liked it. It made some buzz with clients that I worked with. Now I wonder why created those things. I thought then, there must a model using which a testing group can be called mature or immature. The word mature was equated to "Good",  "Efficient", "Desirable" etc. I understood now that maturity is not about good or bad - its about ability to sustain and adapt with change. No model I know of and the ones I created took this approach to maturity.

Another way to look at maturity is how we deal with people. When we say about someone that he or she is mature - it means that person can deal with adversity better, can behave/react with patience and so on. We should apply same idea to software testing. 


Recently a friend of mine bought this idea and rekindled my thinking. Hence I am writing this post.
Most valuable suggestion when I was working my testing maturity model came from my mentor Michael Bolton – who suggested a remarkable thing about the idea of “maturity” (in general). I am going to expand on my renewed model of testing maturity on this interpretation of maturity. Michael suggested that one of the useful ways to define maturity to software (and testing) is to draw parallels with the idea of maturity in biological sciences. Charles Darwin in his theory of evolution – defines maturity as ability of species to tolerate and adapt to the changing surroundings. We all are familiar with tag line of Darwinian theory “survival of the fittest”.
So – my definition of testing maturity draws from this biological sciences idea – testing is considered as mature if it successfully adapts generations of changes happenings in its environment (business and market environment) and retains its relevance/importance. How do you identify such testing practice? Stakeholders are willing to pay for it (challenge me – if you find this statement problematic)
Let us now look at deeper. I think the idea of testing maturity can be applied to a specific “Testing team” (a group of people operating under a corporate structure) or a function or task that needs to be done as part of software making (simple term than saying SDLC that takes me to many other detours that I would like to avoid now). The software Services industry, System integrators, Big consulting companies would like to apply this term to “Testing Practice”. Though the term testing practice sounds very professional (likes of Gartner, Forrester would love) and appear to include both team and function – on the ground – it mainly implies team, structure and some rule book. In most of the cases, software testing maturity is applied to “independent” testing groups – needless to these groups want a label of “mature” so that they continue to live and get funding. Also note that aspects of maturity as it applies to team/structure and to testing as function are not mutually exclusive – there are some common elements.  One reason that I want to make this distinction is that many aspects of maturity take a different shape if I look at testing as group or structure rather than testing as something that a specific team does. You know where I am hinting to. Yes – Agile and DevOps world of software making.

Testing maturity as applied to team/structure
I look at Testing team maturity in terms of Leadership, Doers and testing culture.
A mature Testing leadership would ensure that testing team is responding the change in the ecosystem in which it operates and adapting itself to survive and succeed. A mature testing leadership brings about changes in the team as required and develop collaborative partnerships with developers, project managers, production support teams and stakeholders. A mature testing leadership would not hold its principles and policies as something cast in stone. A real test of maturity of testing leadership is when stakeholder question very existence of testing as a service that a given team can provide. Most of independent testing team have faced this test. A mature testing leadership would be more than willing to break the corporate structure of test team and will be ready to mixed or morphed into any other emerging structure of the organization – an act of self-sacrifice.  Call your testing leadership as mature if it can dissolve itself (the team structure mainly) for the larger interest of testing as function.
Let us now come to “Doers” – I deliberately use this term to indicate group people who do testing rather than the ones who “manage” or “coordinate” testing. Mature testers (doers) focus on constant learning and do not identify themselves with any specific domain, technology or tools or process or like. Mature testers understand the value of adaptation to changing ecosystem and work on acquiring skills to remain relevant in emerging situation. A mature tester thus can operate as effectively in any circumstances and be useful towards the goal that the broader team is pursuing.

A combination of mature testing leadership and mature tester gives an ability of “quick” yet thoughtful response to “change”.  James Bach characterize an expert tester (sorry If just moved from a mature tester to an expert tester – stay on. I hope to establish a connection) as someone who can test under any circumstance of time and other resources.  This ability to test “well” under any circumstances is what gives tester and testing leadership a crucial edge and ability to survive. Isn’t, thus a key aspect of maturity?
Finally – the culture. This is something that mature leadership and mature testers together demonstrate when they are in action. A mature testing culture does not whine about changes but strives to change itself to adapt. A mature testing culture manifests itself in terms of beliefs, collective thinking and set of written or unwritten rules about how testing should be conducted. On any question related to any tactical or strategic aspect of testing – testing culture helps testers (and leads) with “default” response. If watch a team of testers in action – you can distinctly notice the “culture” – if you cannot then probably the culture has not set in yet.
As testing as function continues to evolve and becomes something that needs to get done as part of software delivery – it would be appropriate to turn focus to “mature tester” – an individual. Here too, my definition of maturity is on the lines of “one who can continuously adapt to changes in the environment and evolve”.  Are you a mature tester ?

Saturday, April 01, 2017

Managed diseases and Failure of science

[off topic]

In my opinion, about 70-80% of ailments or diseases treated by doctors using so called evidence based medicines are in the category of “managed diseases” requiring the patient to take medicines life long in additions regular tests and medical consultancy. A very small portions of diseases today are actually curable. This is in spite of spectacular progress of science and technology. From knowing super fine structures inside atom’s nucleus to genetic code, from nano medicine particles to mechanical  heart, from feeling robot to cloned animals - we are the peak of our knowledge than any other generation or human race in the past. Yet more than 3/4 our diseases are incurable and we have darkness at the heart of intense light.  Why science and technology is failing to lift the sufferings of people from these “managed diseases” ? Why science has come to a poor second to nature and life?

I acknowledge role of science and technology in dealing with threats on the life from outside - like accident, fire, suicide etc. Tools and Methods of science have been life saving. Probability of saving life of an accident victim are significantly increased over last 100 years. That is really commendable job of science and medical world.

Coming back to managed diseases - why should we go to a doctor if he cannot cure a disease that has come from inside human body - likes of diabetes, blood pressure, asthama, thyroid and deadly cancer and AIDS? The experience of those who meticulously follow doctors prescriptions is not better barring few edge cases. People lose money, mental peace and suffer through pain while blindly believing modern science and evidence based medicines. Doctors on the other hand blame poor patients that she could not keep up with diet or exercise schedule. When a diabetic patient is about lose a toe due to high sugar levels - doctors would say, patient did not keep the sugar levels under control. Poor patient all that he can do - walk 1 hour day, forget sweets in the life, no fried, non veg or alcohol - probably lived only on salads or veg stuff. Still suffered from all consequences of this glorious managed disease.

Commercial angle of making money through these diseases - pharma, big hospitals, medical equipment manufacturers,  Doctors and institutions that produce doctors and all connected eco systems - is  difficult to miss.

Making money is fine - but cure the disease.

When any alternative medicine or mystic claims some cure - entire world of so called intellectuals, rationalists and supports of science/evidence based medicine pounce on that method and finish it off. Media plays hand in hand to portray anything other than “science” is essentially bad and unreliable.

How can we flip this ratio of managed diseases to curable(time bound) diseases ?  Can science accept its defeat humbly and make way for unconventional methods or new thinking about life ?

Sunday, March 05, 2017

There is no such thing called "Agile Testing" - Part II

My slides of ATDAsia keynote on this topic are here

Here are few key points that I have developed since part 1 of this topic.

1. The problem with current "Agile" is it is stuck and dying its death - in rituals and ceremonies. So called consultants and experts of "Agile" - appear to be pushing rituals and ceremonies without explaining the context and meanings behind them. I find it is very surprising to see people feel proud about following rituals in this rationalist, objective Engineering discipline.  Do not you find this term "rituals" as unacceptable in our field of software that stands as epitome of human knowledge ?

What happens when you do not know the reason and purpose behind a ritual and simply follow it? One - you will apply it wrongly or apply it (the ritual) correctly to wrong situations. When you do something as best practice - you forget the context in which the practice worked and how same or different is your context. The aura of best practice and cult of expert - just blurs your thinking and you get hypnotized. That's where problems start in Agile implementation.

2. There are many good practices in Agile - sorry -  practices that have emerged from the kitchen (not factory) of Agile. These are excellent examples of how smart people have solved the problems in their context. If you understand the context and how problem/solution aligned to the context - you have fair chance of learning, customizing and using the practice to your context. I find practices like lean documentation, dev/test pairing, continuous integration, focus on delivering working software, emphasis on right distribution of automation across technology layers - as good and worth studying. If you start asking - best practice, best tool, best framework, you will miss the background and end up in applying a practice wrongly.

3. Most agree on one thing about Agile - "culture". If you want to make Agile work in your context, you need a cultural change regardless of what is your current culture. This may sound counter intuitive - but it is true. For Agile to work you need culture change.

Here is my prophecy about Agile and Culture - "The culture change you are seeking for Agile to work IS NOT GOING HAPPEN". What is the basis for prophecy? I think culture is made up of people working in groups following rituals while setting aside mostly -  rationality. Humans are lazy, unpredictable, fearful, greedy. Humans want to make profits continuously through software. While not fully understanding "intelligence" - humans have set their eyes on "artificial" intelligence as future. Human for problems in culture - seek solutions in processes, frameworks and tools.

If you want Agile to succeed - take these problematic humans out of equation - with them goes need for this trouble of changing culture. Can you ?

What do you think let me know

Saturday, February 25, 2017

Coaching Testers : An approach for finding answers

Often, I get mails asking testers and budding testers asking questions and seeking my answers. Some of them are questions to something I wrote on my blog. Most of the questions are in the form "what is xxx" or "how to do yyy". 

Here is my advice/suggestion on how one should approach getting answers to the questions that they have on a given topic (this applies to any quest to know something).

Before I answer a question - I will ask you - what do you think? how will you find out? what information or facilitation you need to find answer to this question.

This is how James Bach challenged me when I used to ask him questions in the beginning. As James kept on pushing me back - I realized I must do some homework before ask. In the process, I learnt to find out myself some hints or pointers to question that I have and then seek help by asking "Here is a question" and "Here are my initial thoughts or pointers to this question". "Here is what I find contradicting or not-fitting in". "Here are the sources of information that I used". 

Most of the times - through this process of figuring out, you will get answers in 2-3 iterations without any external help. In this process of finding out - when you are stuck, ask yourself, what information do I need? how will get that information? 


Give it a try - you will learn to find answers to your questions yourself - that would be a fascinating journey.

Saturday, February 18, 2017

Automation takes away Jobs - A reality check

I am not talking about "test automation" here. There is media hype sweeping across these days on jobs being lost, people being fired, retrained on "cutting edge" technologies, re-assigned to new technologies etc . This quora question is an example of people's interest in this.

Let me do a deep dive into this topic

Its a media hype and sponsored Propaganda
If you read carefully into all such reports and media articles and some logic, analysis - it becomes clear that there is a hype and some group of people with vested self interest have been spreading the news. Most of these articles conclude with a call for the readers to do something to avoid "job loss" or any similar harm happening to them due to automation. It might point to learning some so-called "new tool" or "technology" or "take up a course (paid)" or "get a certification". So, commercial interest is apparent. For media, scaring people on some future danger has been a favorite tool to get its end meet. Be it in health care, business or Politics - spreading news about doomsday has worked well for media to form larger public opinion and even make public take actions. People rush to get themselves vaccinated or or buy a term insurance policy or get a health checkup or Hit Gym (commercial interest again) or take a training course - all such actions have a media negative propaganda in the background. As humans, through evolution we have in our blood, an affinity towards negative or bad news. We are likely to believe a prediction of a bad news than a more compelling good news. Media, Sales and Marketing folks exploit this. Can you see this in the tales about job losses through automation? They will scare you to core. When one is scared - rationality and judgemental faculties of human brain are at lowest level. Thus a bunch of scare folks first form opinions about a theme and almost act as expected by "scare-mongers".

What kinds of job are at danger through automation?
As compared to factories and manufacturing assembly line jobs needing human physical effort in addition to some cognitive efforts/skills - IT/Software jobs are/were considered as white color or brainy jobs. In IT and Software - jobs involve varying degree of human elements and intervention. Geniuses in IT services world, riding on outsourcing wave invented so called "low-risk" non strategic tasks such as  data entry and management.  These jobs were defined such that it merely required humans to follow some predetermined SOP (standard operating procedure) in a business process. When there is cost pressure, clients would ask service provider to bring in efficiency. How can one bring efficiency in such brain-dead jobs? Explore the option of reducing humans doing job that can be efficiently done by a machine or a software program. Enter "automation". Look around your business or place where you work - what are those jobs that do not require human intelligence and empathy? If you find such jobs - you can see them going away and given to robots of some sort.

In terms of software technologies side - people say older technologies are going away.  IT services companies providing outsourced technology services will need to support old technologies as long client pays for it. How long client will stay with old technology? That is a business and political question related to a client's business. Typically there is a huge cost to move from a legacy tech to a new tech - its is called "Migration" or "Re-engineering" program. Since such a "change" involves new learning for the staff, new infrastructure and cost of development/migration - businesses tend to stick around an old tech stack until a point when it absolutely becomes impossible to continue. When did businesses move from Windows XP to Windows 7 as desktop operating system ?  Around 2013 or so Microsoft announced end of support for Windows XP. This is an example of technology upgrade. As an individual - if you are stuck with an outdated technology- watch out.


Is this new?
What do you understand from the term "digital"? If it was early 90's - it would mean anything done using a "computer". Year 2000 onwards - it meant something done using internet. In last 6-8 years, it means "mobile". But at the core, in computing technology - the phrase "digital" compares with "analog". When did we last hear about "analog" computing devices? I had nice fun the other day arguing with a colleague on internet is as "digital" as mobile. She believed that qualifier "digital" applies to only "mobile". What will happen if quantum computers make way into mainstream computing - will those computers be called as digital?

Going digital for a business mean, in simple sense, a part or whole of business involve "mobile technology". This shift from desktop computers to internet to now mobile - has been causing many traditional jobs that were performed with "digital" technology - to go away. Just like digital camera era killed likes of photo film maker - Kodak.

Media propaganda makes one believe at first that such job losses are unprecedented and happening for the first time. In the past too - when computers first came, people who resisted them lost jobs as in some sense computer did the work better and cheaper than the humans. Some intelligent ones immediately re skilled themselves and embraced the change. These folks not only survived the technology change wave, some even flourished like never before.  Like biological evolution, business constantly keep looking for ways to make more money given constant or reducing capital and resources.

Your career is your responsibility
Software job, fortunately or unfortunately is not a job covered under an employee union (by and large there might be exceptions). When your company fires you without giving proper justification - you cannot knock some outside entity to get you reinstated. Businesses world wide using so called skilled and white collared jobs - can take liberty of downsizing workforce should going gets tough with falling revenues and profits. While on job, keeping one updated with skills in emerging areas of technology and business - becomes responsibility of the individual. 

In Infosys related quora post above - mentions that affected people are trained in "cutting edge" technologies. I ask - why do people do or get stuck in "blunt" or "old technologies" in the first place? Why do these folks (if at all they do) want their companies to take care of their careers or skills? Why cannot these folks keep improving the skills based on emerging market conditions? If a company displaces people working on a "blunt" technology due to low or no demand - should you blame the company? While keeping people working on some outdated technology might be a business imperative to companies - getting stuck in outdated technologies with or without knowledge at individual level is detrimental to one's career and society at large


 If you are happy with 9-5 cool job that does not require you to any great deal of application of skills or knowledge - be ready to have your job redundant any time. When jobs that do not require skills are lost - media might make noise about this. Again - if you see the vested interest behind these, it becomes obvious that it is an attempt to form public opinion in a specific one way away from the reality. You cannot depend upon your company to keep you in front-line tech or business work all the time. Its your job to be good at what is in demand and then have company to keep on fore-front.

When you hear "automation takes away jobs", ask "what kind of jobs" and what you are supposed to do ? Watch the reaction and share it with me. You should be able to smell vested interest behind such a claim.  Would you ?

Friday, February 10, 2017

Two important lessons for success of Test Automation

James Bach wrote this great article on how not to think about Test automation way back in 1999. Anyone starting into automation and those wanting to learn more about automation - must read this article. First of all automation is about testing. If you think narrowly about testing - your automation will be narrow.  Even today it is not uncommon for business leaders to say "do not have time or resources for testing - do automation". I hope some business leaders in IT, Software, Testing are reading this post and make amendments in their view.

I would like to put two key lessons that I learned in these years that you can use to make most of your money you are putting into automation

If a test (case) can be specified like a rule - that MUST be automated
Automation code is software - thus, obviously is built on some kind of specification. Most GUI automation (QTP, Selenium) is typically built based on so called "test cases" written in human language (say English). It is the first question that a automation guy will ask while starting automation - "where are the test cases?". In dev world - automation takes a different meaning. In TDD style automation (if you call TDD tests as automation) - test is itself a specification. A product requirement is expressed as a failing test to start with. The approach of BDD throws this context to other boundary, specify tests in the form of expected behavior. So, automated tests are based on specification that is a human language but expressed in business terms (mainly) and with a fixed format (Given-when-then).
Key lesson here is - if a test can be specified like a rule with a clearly defined inference to be drawn from the test - that should be automated. Automating a test means create a program to configure, exercise and infer results of what test is trying to validate. Michael Bolton calls such a test as a check - a meaningful distinction. If a test has human element in it for inference mostly - you cannot possible automate the test in its full form.
How do you implement this lesson in your daily life as tester? When designing a test - see if you can specify it like a rule.  If you can then explore ways to write a program for it. Then that test becomes automated. In this way when you are building a suite of tests - some are specified like a way that makes it easy to automate and some are specified in a way that a human tester need to apply her intelligence to exercise and infer.

Automated tests (checks) are like guard to product code
A child asks his father "what is the use of brake in a car". "it helps to stop the car" says father. Kid responds back "no.. I guess break helps driver to drive the car as fast he wants to as he has a means to to stop when needed". On the similar lines - having automated tests around a piece of code - literally guarding the code - empowers the developer to make changes to the code faster. More often than not - bigger speed breakers for development is fear of breaking some working code. Developers are mostly worried about large chunk of legacy code that one rarely understands fully. Having automated test as guard - what happens is test will flag change in the code via failing test. Armed with support of guarded code - developers can now make changes faster and can depend on tests to tell them if any of change made has broken some other "working" code.

How do you implement this lesson? Work with developers and help them creating tests that guard their code. These tests should work like "change detectors". Writing test automation would require knowledge of product code and principles of unit testing. Not for weak hearted GUI QTP/Selenium folks.

Thursday, December 01, 2016

Test Manager vs. Project Manager : Changing motives and Perspectives

I am running a project as test manager - typical one, every day changing requirements, committed go live date, huge code churn, week ends included in schedule - this project has all fun, frustration, excitement - emotions galore.

As a test manager - what drives my energy is to find problems and report them in best possible way. Honestly - seeing more problems in the application drives me and my team. We get excited if more bugs are discovered. We celebrate every new find and I can see shine faces of my team members. Any news of erratic behavior, application crash, instability of code, environment down - makes us feel happy. Often I think, are we testers sadists?

Sitting next to me - is my friend, colleague - the PM. He is worried man. Every time someone in my team stands up and asks for some clarification - this PM's heart beat goes up and must be thinking - oh no... one more bug !!!  During our bug triage meetings, I speak proudly "40 new bugs today and that makes this week's overall tally of 370, 80 of these are critical". My PM friend after regaining calm says "ok - how many fixed bugs are retested? which areas of application are relatively stable? what positive news we can take to our stakeholders".

See the clear change in perspective?  PM wants to see what is working, working fine, what positive news we can report? Test manager wants to boast on what new problems testing team has found.  It makes sense for testers and test managers to get into shoes of PM's or Dev team once in while to understand what these folks think.

While tester should not lose their sight on finding problems and making sure that they are reported well - collaborating with PM/Dev and stakeholders to achieve a convergence of code towards release/golive date, can often be very useful for over all project stand point.

More often than not - due to changing requirements, unstable code, challenging deadlines - except testers, everyone in the team lose sight of golive. It is like being in a tunnel with no light from other end. PM's and Dev team would be watching with clenched fists to see the end of testing cycle.

The friction between Dev, Test and PM often is due to this differences in perspectives, motives and lack of communication on big picture on Go Live date.

Dear testers - when you find yourself in such situations - show empathy towards fellow team members. Pause sometimes and ask - can I see the project from their eyes, what are their worries and how I can help.

This will go long way in good team bonding and you will be called as "mature tester"

Sunday, November 27, 2016

Tester or Leborer ?

A friend of mine sent a link to this article on PMP and Project Managers that brings out an aspect of our profession - testing so beautifully. Are we knowledge workers paid for our expertise or laborers?

How does whatStuart is saying about PMP and Project management apply to Testing? I believe, more than certification, testing profession is hit by the way we poorly define testing and adopt a model of testing that eliminates need for skill, focuses on mindless repetition of some documented procedures.

Time to reflect on. If we define and accept that definition of testing that systematically undermines skill element and focuses on process, tools, metrics etc - there is no doubt that we will become laborers.

Is testing rule based?

How much of good testing is rule based?

Saturday, July 09, 2016

Testers are human ... so are Programmers

One important aspect of we humans as testers or programmers is how our day-today happenings impact our work at office. While this is not different for we software folks as opposed to any other profession that requires "presence of mind" - being occupied with thoughts about past or future can lead "knowledge workers" to make mistakes and/or forget things.

An incident that occurred this evening made me realize how important is for testers to be "present" while testing so that do not miss things and make mistakes. I went for a shopping mall with my family nearby. While entering I had an argument with a fellow who while reversing the car in parking happened to hit my car. While that incident fresh in my mind - I passed parking ticket counter, collected the ticket (while my mind of full of the car incident little while ago) and gave it to my wife. I generally has a designated place in the car where I keep all these tickets. This time my wife kept the ticket in a place that I generally cannot reach from driver's seat. I did not mindfully record nor my wife remembered clearly where she kept the ticket. Few hours passed by. While returning, my wife and kids went a nearby place and asked me to get the car and pick them up,  While walking back to parking lot - I was confused about where was parking ticket - thought of parking ticket was all over mind. When I reached car, I searched my usual places, did not find the ticket. I panicked on the prospect of paying almost a day's parking fee instead of few hours. I did few more rounds of check around drivers seat, usual places that I keep ticket and pillion seat - did not find the ticket. Finally called wife to check if ticket is with them. I was told that ticket should be in car. I finally gave up and paid the full day fare and came out of the parking.  When my wife comes in the car, she reaches out to glow box at pillion eat and hands over the ticket to me.

Why it did not occur to me check in glow box ? Why my blocked mind did not contemplate on various possibilities and locations for the ticket after all car is not such a big place? I guess two things happened. One - due to argument with other driver at the mall entrance filled my mind so that I did not mindfully register where my wife kept the ticket and second I gave up easily before exploring my options.

What did I learn from this incident that I can apply testing?

Good testing is about having wide range of testing ideas to cover mistakes that other folks do while constructing software. Programmers, Business analysts and others can make mistakes like I did. I urge testers to be mindful while testing, designing test cases and watch out for mistakes/misses that might lead to bugs. Every now and then put your mental abilities about test idea generation to test and develop the skill to look for misses/mistakes. Practice mindfulness and be vigilant at all times. This helps in your personal and life outside office as well - you, yourself are least likely to make mistakes.
This will save time, money, rework and will give you peaceful life. What more - you can do the same for others.

Role of human emotions in software development and testing has been the point of discussions at many testers meets and conferences. I guess the larger software community needs to acknowledge this and develop measures to be mindful.

I suggest mindful meditation and concentration exercises to testers having high level of mental activity (more often than not - nose) - like me. Being mindful and vigilant at all times - seems to be now a skill and capability for testers.