Thursday, June 08, 2006

Adaptive Automated Testing ....

I came across this topic on automation

Most of the White paper looked like a sales pitch for the Aberro Product but this seems to be a new and interesting concept.

A brief review by me ----

Objective of automation

 Provide high coverage of the application under test, cost effectively --- Accepted
 Enable early deployment in the development cycle when defects are less expensive to fix --- Does not come under Traditional Test automation that we know today. I would rather apply extensive skilled manual testing and reviews to limit the defect early in the cycle – Not by using automation.
 Be fast and inexpensive to develop and maintain tests – Fast - yes but “ inexpensive to develop” will potentially remain is “Forever wishlist”
 Eliminate the requirement for programming skills – Very incorrect advise and nearly impossible goal to achieve
 Adapt well to changes in application functionality - Very Good one and somewhat likely to achieve
 Enable fast, unattended test execution – Accepted and achieved in most of tools available in market today
 Provide strong verification capability – Verification capability is a strongly related to Testability of the application. Most of the tools in the market today don’t have this as key functionality

Comparison of Various Automation Techniques
Page 11 – good one.

What Adaptive automation means –

1. No Test authoring – No manual test cases required for automation
2. Can be adapted in any phase in development cycle – even when the application is unstable.
3. Almost in-sensitive to application changes – Tool adapts to the application.

I am not sure how it works, appears to be interesting.

Finally few Not-so-good points in the paper

1. The paper title has keyword “Automated Testing” instead of “Test automation” – there is a difference. In no sure terms, testing as an activity can not be automated. Only execution part can be automated. Anything that talks even remotely about “Automated Testing’ in software world is a suspicious and is a sales pitch.
2. Paper starts off selling “Testing” by quoting some famous quote describing 60 billion dollar loss by software defects. Do we need to sell testing by talking about some survey done in 2002 and by talking about defects? I am sure there is better way ….
3. Mentions that “Manual Testing is labor intensive and hence the cost” – So is quality. Even development is labor intensive – why there is no concept of “Automated development” – one that sells? Saying manual testing is costly hence go for automation is a bad argument in favour of automation. Further equating manual testing to “brute force” – is to insult the craft of Manual testing – I strongly object to this.
4. Mixes up QA and Testing all throughout the document.