Sunday, July 26, 2009

Tom DeMarco’s confession

Confession is probably a harsh to describe what Tom DeMarco, the creator of celebrated punch line of software managers – "You can't control that you can't measure", wrote recently. But, if you read Tom's recent article "Software Engineering – An idea whose time has come and Gone?" in IEEE software, you would probably say something similar to this. In this short 2 Page article, you will find Tom DeMarco in reflective and retrospective mood.

In the book "Controlling Software Projects: Management, Measurement, and Estimation (Prentice Hall/Yourdon Press, 1982) ", Tom talked about "Controlling" and "measuring" in Software Engineering. Nearly 40 years later, he now appears to admit that he pushed the notion of "control" and "measurement" too much.

This "confession" has apparently caught the attention of many. Jeff Atwood writes an obituary to "software engineering". More than his post, the comments for the post of Jeff Atwood are interesting to read. How come, suddenly so many are accepting now that "software" is human centric, people oriented, "engineering" is not the right term to use and so on? Michael Bolton's recent article (three kinds of measurements and two ways of using them) on stickyminds appears have been triggered by Tom's "confession". Matt Heusser writes about metrics here, here and here.

Managing vs controlling

In his own admission, Tom seems to distinguish between controlling and managing. He gives the example of "upbringing of a teenager". As any family therapist would recommend, you manage your teenage kid rather than controlling them. Like in most human endeavors (including software programming and testing), you can manage quite lot things than controlling them. I think we can manage (and also a control a bit) WITHOUT measuring ANYTHING at all. I like Matt Heusser's example of hair cut. Many of us control and manage our hair (style) without measuring.

Manage people and Control Money and timelines

This is Tom's recipe for managing the project without controlling it. While breaking project into human elements and non human elements (such as code, schedule, money) is a welcome change, I am not sure how can you do it – managing people and control money and timelines. To me, these two sets of things are NOT mutually exclusive so that you can treat each in a different way. Controlling time and money impacts people and managing people impacts timelines and money.

Some software is really engineered!!!

Dave Markle makes an interesting point in Jeff Atwood's post - "IMO you can't say that programming languages themselves aren't engineered based on solid computer science. You can't say that something like LINQ hasn't been engineered. Whenever you use a FSM in your software, you are applying computer science, which makes you a software engineer"

Programming languages are engineered, operating systems are engineered and so are software algorithms. It probably the user mass/ size of the group decides engineering vs. crafted.

Dave uses the Paint analogy nicely to drive home the point - the paint artist uses is engineered (developed and mass produced using the principles of chemistry and physics) where as the "art" produced by the artist is NOT. This stirs the nest of debate of what is engineering and what is craftsmanship. Are Engineering and Craftsmanship are mutually exclusive? I am afraid NOT.

Tom, Damage has been done and is still happening. Many still many abuse your punch-line to push loads of documentation, process, approvals, and meetings and of course endless charts/graphs of metrics – all in the name of "control". Probably the time has come to step back and be sensible on "measurements" in software.



Michael Bolton said...

My article on measurement, "Three Kinds of Measurement and Two Ways to Use Them", on StickyMinds, was written several weeks before Tom's article was released. Their appearance at more or less the same time was a happy coincidence.

---Michael B.

Michael Bolton said...

Note also: we do measure our hair length. But we use first-order, qualitative measurements, rather than second-order quantitative ones. "It's too long" is something that we can say irrespective of what numbers tell us.

---Michael B.

Kashif Ali Habib said...

Nice post.
this post gives me a new thinking direction, about difference between managing vs controlling.i will definatly drill down this topic.

Anonymous said...

Nice article...if you enjoyed reading this article, you may also enjoy reading-

Rajesh.Ediger said...

after reading this post there are some doubts i got about the measurement and analysis, as am into Quality assurance we do collect metrics data from projects to do analysis. With some data we can make some decisions otherwise it is all assumptions we have to make and take decision.

Shrini Kulkarni said...

@Rajesh -

"With some data we can make some decisions otherwise it is all assumptions we have to make and take decisions"

My suggestion is not use metrics to make decisions. Use metrics to probe the idea/number/story that is put forth. Use metrics to ask questions like "What else could be true"? or "What would be a totally contrasting explaination"?

Please note - metrics are bare numbers - people using the metrics can come up with potentially many stories explainations...

I typically ask when I am shown a metrics ... "how many stories are possible for this metric - now" ? Which one I should probe first? how can I unearth the multiple dimensions that are burried in the number?


Naga S Dandu said...

True Numbers doesn't show the Facts , but how do people measure the progress of the project with out Number ....

Naga S Dandu said...

True there are many stories behind the Numbers , How do people measure progress of project with out Numbers............