Other day, some one asked me “in your opinion, what are the real benefits of automation”. That triggered ideas for me to write this post… Let me attempt to consolidate and list all the benefits people claim around “automation”
Real: In my opinion, these are “real” and achievable benefits.
- Consistent and Accurate Test results (nearly free from human errors) – when accuracy and accurate results are important? – Numerical calculations
- Untiring and can be run for long hours without any loss of efficiency
- Quick – No think time while running tests. (Computer program does not think – So not useful in those cases where you need to think as you execute. How does automation program help you to gain speed when thinking is required?)
- Supplements human ability to spot software bugs
- Helps to run big number of data combinations – can test robustness
- Helps in multi platform combinations (OS/browser/database and other application setups)
- Repeat the testing (test execution) done for configuration A in Configuration B also.
- Hence increased Test coverage
- Following non test execution tasks
- Generate and manage special set of test data
- Large Volume Test comparisons
- Automated workflows and alerts
- Environment setup
Fake: These benefits are on transition point – where the focus slowly starts drifting from “real” to “imaginary” and hence “fake”. These are false promises that are realistically not possible.
- Improvement in application quality (automation can not improve application quality – even testing can not … only developers and business analysts can)
- Improvement in Test process (Test process is pre-requisite for automation)
- Non technical people can do automation – no programming language required
Improved test planning (not sure how)
- By executing a set of test cases for a specified number of times – ROI from automation can be realized. Say in 10 executions –automation test pays for itself after that is “cost saving” all along
Conditional: These are the benefits that are realizable and could be reasonable but under a very strict set of conditions – “once in a while” cases. When these benefits are stated, mostly the conditions that must be fulfilled to realize these benefits, are not stated. This makes these benefits look as though there are real and universally applicable
- Simply – saves time.
- Test effort reduction
- Improved Time to market
- Improved test productivity
- 24x7 testing possible
- Knowledge retention
Totally outrageous: There are some really outrageous and are more or less like a typical sales pitch. People, who make and believe such claims, do not seem to understand human side of testing, testing itself and automation. They are just believe that machines are better than human testers.
- An automation test execution is equal to an hour of skilled human testing
- Automation can replicate human interactions
- Reduced dependency on human testers
- Solves problem of resource crunch and less time available for testing
- Reduced defect rate – fewer defects
Anything I missed?
5 comments:
You should reference James Bach's list of Reasons to repeat a test as well. But nice list so far. I hope I have the time to look through it at home.
Regards Markus Gärtner
My project performs automated test executions on daily builds everyday. These are the very basic end-to-end system test scenarios. This most important benefit we've got from this practice is 1) ensure if testing team need to pick the new build for testing 2) In case a defect is detected in the execution, developers know with enough certainty "when/how/who" of the defect.
@Rajesh ...
Yes, there are automated tests that are developed at Class/API level and used for daily builds and smoke tests. These are typical in Software product scenarios.
Are your tests developed at Class/API level or GUI level (I suppose it is the later)?
Automation Smoke tests to check if the build is stable for rigurous testing or not is a another benfit .. that comes under "Real" category.
Thanks for pointing out that.
Shrini
@Markas -
James' List of resons to repeat is a great article to read. Every time I read, I discover new intepretation of the paradigm of regression, repitition and scripted tests.
I am not sure how that list is related to this ..(other than common point that all automation is repitition of some previously executed test.
Shrini
@shrini,
Our test are developed at the GUI level. Developers have their own set of test at the class/api level and they are good at detecting unit and integration defects. However our experience has shown the developer tests often fail to detect bugs in end user scenarios.
Post a Comment