If estimation is harmful… then, what’s not?

Or leaner Agile – part II
(Part I is here)

During conversations with a couple of co-workers about estimation (after writing this), we were all reminded of strange situations where estimation went haywire. Here’s an example. I was with a small team at a potential client and we had been creating a proposal for a fairly simple project. We intuitively felt that it would take the four of us about 3 months to get done and our high-level estimates validated this guess. However, the project-management team wanted us to break things down into a more granular level, to ensure nothing was missed and so on. Which was fine, and so we did. When we re-estimated the smaller, lower-level tasks, we ended up with a total of 8 months for the same team. No one could really argue with that, the cards were all laid out on the table. We didn’t get the project, a competitor came in and did the project in about four months.

I’ve heard several war-stories that talk of how people break things down, analyze them to death, and then estimate for worst-case scenarios. Then they pad the estimates, sometimes at all levels – at the task-level, at the story-level, and at the project-level. This simply bloats the estimates to a bizarre level. In the case where the estimation is being done by an external vendor, they often risk losing the work. When this is done by an internal, and often captive group, the customer is left with little choice. They budget for the bloated estimate, get the money approved, and end up spending it. After all, no one ever got in a fight with Mr. Parkinson, and won.

We also talked about times where teams spent more time breaking things down and estimating them than they spent to actually implement the darn thing. All fun stuff.

These stories make me think of Heisenberg’s uncertaintly principle – the very act of trying to measure something seems to change it. Obviously this doesn’t quite fit what we’re talking about here, but still… I’ve seen people break things down into minute tasks – and then when they begin to assign time-based estimates to them, they still go with fairly large units of time. Maybe half-day units or sometimes a couple of hours. And then, next thing you know, 2 or 3 simple tasks take up an entire day…

Of course, in certain situations, there is no option – you just have to break things down and make your best guess. This is especially true for vendors bidding for projects. When you’re not in this situation, however, there really is little benefit of taking this approach. Collaborate with your customer instead – do high-level estimates for baseline sizing, get a team together and get your hands dirty, then make claims about when you might be done. This will definitely work better – and will allow the customer to be more involved in the whole process – and they’ll be able to give feedback about what they really want, based on their involvement with the evolving software. In other words, keep it simple – just use your velocity to determine when you might be done. If I have 700 points of work (high-level estimate), and after four iterations I’m doing about 60 points an iteration – I think I’ll be done in another 8 iterations! Quite simple!

And by all means, revise your high-level estimates ever so often if you so wish to do so. Every time something changes or a risk becomes a reality or scope changes or your architecture group changes direction – take another pass at the sizing exercise – and derive a new answer. Just keep it quick and simple, your answer will probably be close enough to the answer you’ll get with more detailed (and more painful and more expensive) analysis.

The trick to this, of course, is to step away from time-based estimation. When people think of time-based estimates, they begin to practice defensive-estimation – and pad estimates “just to be safe”. This is done almost unconsciously, and also quite casually, so much so that it seems perfectly natural. And it adds up. So, by using story points instead, one can easily side-step this issue. The theory is that although the end goal still is to answer the question “what will it take (in terms of time, resources, money)?”, using story-points splits the answer into two parts. The what, and the how much. Story points address the what – by only focusing on the relative complexity of the work. It also has other advantages. The second part of the answer manifests itself after a few iterations – you divide the total number of estimated story points by the team velocity to get an approximate duration. Again, pretty straight-forward.

P.S. – Finally, buffers have their place, since a system with no slack can’t cope with change. However, that is a topic for a different post.
(Part I is here)

9 thoughts on “If estimation is harmful… then, what’s not?

  1. I personally like the way Mike Cohn wrote about it in his “Agile Extimating and Planning” book. Chapter 6 is all about trying to acheive the correct balance when estimating. There is a graph that shows clearly the amount of effort put into estimating will be less accurate if too little effort is expended and is also shows a dimishing return if too much effort is expended.

  2. We use detailed estimates for all development tasks. Why? Well, too often your gut feeling of “3 months, give or take” will be received with “ok, do it but in 1.5 months”. Arguments like “umm, my estimate was based on 14 years of experience” usually falls on deaf ears. Regardless, it will be me and my team doing the overtime in the end…

    To that end a detailed estimate, based on a detailed brief and a response-to-brief document where you identify everything that you can foresee there and then goes into an Excel spreadsheet. If the brief changes, so does the estimate. If the answer still is 3 months, and they still come back with “you have 1 month” you can cooly go to your machine, print out the spreadsheet, throw in front of them and say “ok, you tell me what I’m not doing”. In short, it gives you more ammunition. It also educates the non-developers on what actually goes into a project.

    We then summarize everything down into hours. Those hours are then spread out over 6-hour days for the main project all developers are on, thus creating slack for meetings and other admin tasks that also are part of a normal work day.

    Works perfectly for us.

  3. Mathias obviously has practical experience in the matter and he’s right do what he does.

    That aside. I find that with work estimates, experience and best guesses are only as accurate as the weakest links. As you flesh out the requirements, take time to identify areas that’s you aren’t as comfortable with — technologies that you have little experience working with or are unsure about the implementation time. These are the places to spend time breaking them into hourly tasks and padding. Broad brush the items you’re comfortable with and get to work!

  4. Bob,

    I agree. Having both quantitative (hours, points, whatever) and qualitative (assumptions, risks) estimates is extremely valuable. This allows one to have an educated conversation with management/customers around why things are progressing the way they are.

    Also, it makes it easy to mitigate risk – one can pick the top items from this list of issues, and write code/spike tools/whatever to reduce the overall risk.

    So overall, I completely agree. Again the difference in my approach (and what I’ve used on my projects) is around how to project the completion date. I prefer to use real team velocity, instead of the hours and tasks based estimation.

    In the end, agile is about what works in the situation you’re in, and about adjusting the process to suit the project, team and organization.

  5. Another thing to watch out for is group think when estimating time. I’ve seen a lot of times when a list of tasks are on the board and one person says 3 days for their task (I find that if you say anything over 1 day then the task isn’t fine grained enough). Then suddenly everyone else is saying their task will take 3 days as that’s now the default “I’m pulling this out of thin air” number.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s