Estimation considered harmful

Or leaner Agile – Part I
(Part II is here)

This is a rather long post on a topic that has sparked many arguments whenever anyone – colleagues or friends – brought it up. It is about applying a key concept of Lean to, what some may consider, a rather extreme degree. And I’ll admit this – the title is a little misleading, and deliberately a bit inflammatory. Anyway, here goes…

The what

I want you, dear readers, to walk with me. Walk with me and follow along as I travel the path of an imaginary project that we, as forward thinking Web 2.0 visionaries, take from an idea stage through to construction. And during this journey, I want to show you what place estimation as a practice ought to occupy in the ideal software development life-cycle.

So, first we need an idea. Hmmm, a quick look around tells us that a mobile social-networking website would be cool – it would be just the thing to get easily funded, and hopefully, bought by Google in short order. Having satisfied ourselves that this is a solid business plan (the VCs are happy with it), and that it’s (clearly) bound to work, it is now time to get started on getting the thing built.

Now, what features should the site have? Since we are the founders, we brainstorm among ourselves, and since we’re hip, we even ask a couple of potential users to participate. At the end of the day, and after a long and grueling session of arguing about what’s useful to teenagers and what isn’t, we have a nice, shiny list of stories. Again, since we’re hip, we decide to call it our Master Story List.

The how much

OK, how much money are we going to need? And when can we launch the first release? Certainly, our friendly VCs will ask us such questions. Hmmm… time to call in the cavalry. Through our connections, we hire the best developers we can find. We also hire a couple of QA folks. Since we plan to play the role of the business-analysts ourselves, we have a full quorum. We’re ready to do some estimation. We don’t really know too much about the system, at least none of the specific details, and nor do we know exactly how certain things are going to be implemented. What we do know, is at a high-level – what we want built and what each feature might look like. With this input, and with the years of software development experience that the development team has, we assign story-points to each requirement on our Master Story List. After another couple of days of intense sessions (discussing and arguing about what each feature is, how it relates to the market and to other features, and how it might be implemented and so on) we finish our first cut of the sizing exercise. Our story-points have a simple scale (lets ignore unknowns for now) – 10, 20, 40, and 80 – where a 20 was a small story, 40 a medium, and 80 a large. A 10 was assigned to really simple stories. When we added everything up, we had a total of 13,340 story points.

Wheee! We’re done estimating! At least for now. Again, from all the experience that we have as a group, and from our best guess of how things might go on this particular project, we think we can get 600 story points done every couple of weeks. The full project now appears to be about a year in length. We also want to do multiple releases so we can release something quicker. So we pick the features that we think we absolutely need for the initial launch, and it turns out to be about half the overall set. We therefore plan for a six month initial release, followed by several shorter releases. We’re ready to go!

By the way, when we spent those few days locked inside the conference room discussing features and so on, we also derived a list of questions, assumptions, and risks. We’re going to use that list as a starting point for our project-long risk-management activities. We’ve got all of it written down (in a nice Excel spreadsheet, no less) and the plan is to update and monitor everything on a weekly basis.

Go! Wait! Iteration planning

So – we’re ready to start! We bite off a bunch of work that we think we can get done in the first couple of weeks – and the developers go at it. How exciting! Now, we’ve read books on Agile software development, and some of them recommend that at the iteration planning session, we should re-estimate those stories that are planned for that iteration. They say that this detailed, more granular, task-level estimation is a key part of the iteration planning meeting (IPM), or sprint planning meeting. Much of the literature also says that we do this second level of estimation in real units of time (as opposed to the story-points we used earlier) – so that we can plan iterations better, and track progress better.

We think about this – and question the value of this effort. We already know what needs to be built – for this sprint and for the release. More accurate estimates will not change the amount of work, much less reduce it. So we decide to save the half-day or the whole day that the team might spend discussing design and implementation details to get at the more accurate estimates (wow, that’s some oxymoron, huh?) All that stuff changes the minute someone writes the first bunch of code, anyway. So we decide to let the code, and hence the software, speak for itself. We let the developers do what they do best – we know they’ll use good software development practices to ensure high quality. The QA folks are here to test the application as well. The actual working system is what matters, anyway!

We do, however, ask the development team to break things down into manageable chunks of work – in other words, they task each story out. This is useful to them, because it helps them think through things, and keeps them focused throughout the implementation of each story.

Keeping it simple

In making the decision of not bothering with more detailed estimates, we also thought of this – there are only two things that we can play with – scope, and schedule. Quality has to be high – that’s non-negotiable. We’ve hired good people – they’re enthusiastic about the project, and they care about their craft. We’re sitting in the room with them, and helping move things along whenever they’re blocked on things. If we want to get done quicker, the only option we have is to pull stuff out of the release plan. If we want all the stuff that we had decided on, no matter if things take longer than we hoped – the only option we have is to delay the release. It’s that simple.

Remember the old days, before this whole Agile thing, when people thought that the way to control the complexity of building software was to freeze requirements before starting work? That didn’t work so well, did it? Today, another delusion persists in the minds of many managers – that of being able to predict the date of completion with accuracy. Just like the weather (the most powerful supercomputers can’t predict weather with any accuracy more than a few days out, and they often get it wrong anyway), software is too complex to think that by controlling a couple of variables, one can control the trend-line. Even that control doesn’t have predictable results – by de-scoping a complex story (whatever complex might mean to you), is there a quantifiable way to say how much time was saved? Or by re-factoring a gnarly area of code, can you tell how much efficiency was gained?

Why do we still persist in trying to adjust for velocity, drag, complexity, performance, estimates, re-estimates, actuals, rate of scope increase, etc.? Why do we expect that by running massaged numbers we can produce the date? After all, what software team ever announced a date that they actually met? These days, many companies just stick the number of the year at the end of the name of their product – Office 2007, Windows 2003, Pocket PC 2006. They’ve given up the charade – they’re now saying they’ll just deliver it sometime in the year. Just keep your fingers crossed.

So now what?

So then the question becomes, if the software development is going “slow” (in other words, estimates turned out to be just that – estimates), when will we be done? And can we do anything to speed things up? The answer to when comes from the story-points we’d assigned to every story coupled with the data we gathered over the past several iterations. If it appears we’ve been getting done only 450 points each time, then, hmm… we’re going about 25% slower than originally “planned” and realistically, our six month release is looking more like an eight month release. Again, fairly simple math gave us the answer. The second question is also fairly simple – and in fact, it has nothing to do with estimation at all. It is a quintessential question about software development – how can we deliver faster? We can’t answer that – maybe no one can. Doing things right, hiring smart people, these things help. For now, we either accept a delivery in about 8 months time, or we trim scope.

The thing to note, also, is that if we’d spent the time to do detailed estimation (at the task level for each story in the sprint) – we’d have lost another half-day to a day every iteration, and we’d still have no real change in the outcome. Sure we’d have data as to how accurate (or entirely off) our development team was with their estimates, but that would be about it… it wouldn’t change the amount of time they’d have taken. And we’d be short about 5 to 10 percent of the time to actually build the software. Oh boy, we sure made a good decision of not going even slower!

Some books on Agile also talk about how important the planning meeting is… what with the estimation effort, and the communication that goes on during it. If you’ve been following along, we just thought through why detailed estimation is a wasted effort. Now lets talk about the communication aspect.. The theory is that this meeting allows developers to find out what other developers are working on, and also gain understanding about the other areas of functionality. All this as the business analysts drone on about how this feature or that. Didn’t we catch sight of those two QA people dozing off? And that developer? Some of the other BAs looked pretty bored too… the reality is that long meetings are not good forums for anything, much less exchanging information about something as complex as software. When developers are curious about someone else’s code, they bother each other, and talk about it at a whiteboard – our whiteboards are always filled with the remains of all these technical discussions… squiggles that only mean something to the techies.

Re-estimation

The literature also points us to another advantage of detailed estimation and re-estimation. The idea is that if we keep careful notes (and numbers) about what and how the team originally estimated each type of story, then if certain estimates were consistently off, we could go back and fix those types of estimates across the project. Thereby, we’d improve the accuracy of the end date quickly and easily. This makes several assumptions – the biggest one is that the original estimates were all consistently inconsistent. Ouch, now we’re in the land of mirrors. Not only were they consistently off, but they were consistently off for the same reason – a leap of faith, really, given all the variables.

Now, I’ll admit, I don’t see any reason not to try and “fix” incorrect estimates, especially after you know more about the system. It’s just that you don’t need the whole detailed estimation waste-of-time for this (or actuals for that matter – they’re evil also and I’ll talk about them another day). Any good development team will realize that certain estimates were made on an assumption that was wrong (or whatever), and will provide information that will enable management to go back and correct for such things. This is just a normal and an everyday software development activity – you make an assumption and move ahead, when things change, you go back and adjust. Normal communication habits are all that one needs. Some people say developers are not capable of doing this – they only care about writing code. They say they’re geeks and don’t know how to communicate, and don’t know what’s important to management. I’ve even heard that you can’t trust developers to raise visibility of such things cause it’s like admitting a mistake or something. Oh boy – if you have such developers, your project is doomed no matter how good the estimates are.

However, like I said, there is value in going back and correcting your plan based on more information or a risk being realized. Again, this should happen automatically, every day! This is also a happy side-effect of the regular risk-management sessions we talked about earlier. Finally, just like quality, risk-management is everyone’s problem.

Further planning

One more question that invariably gets asked when this topic is being discussed: OK, so if I go with this approach, then how do I know how much functionality to plan for in a given sprint or iteration? I have two answers – the first is to simply use yesterday’s weather – if you got done 500 points last sprint, try the same thing this time. If half your team is going to be gone on vacation, try shooting for 250 points. Again, simple and as effective as anything else.

The other answer is that iterations are evil, they add no value, they’re considered harmful, and should not be done. But that’s fodder for another post.

Anyway…

So this is my pitch about why detailed (or task-based) estimation is in general a waste of time and resources, and is a contributing factor of a lowered velocity (the 5 to 10 percent we talked about, depending on how much time is spent every iteration, more thanks to the interrupted flow) . It also adds an unexciting element to the software development process that adds no value to the actual deliverable.

So, do I believe that all teams ought to abandon low-level (or detailed, or task-level) estimates? No – I believe this is an advanced technique – and can only be applied effectively if all the players in the development team truly grok software development. It assumes that all players are equally enthused by the art of building software, and are engaged in the iterative process of development that is Agile. If your team is just starting, or you have team members that aren’t experienced using Agile techniques, and especially if you have a project manager that doesn’t fundamentally understand agile and/or lean software development, then you’re most definitely better off doing things the traditional way – and estimating at the task-level.

For others, here’s a way to trim some muda off your process. Flame away.
(Part II is here)

25 thoughts on “Estimation considered harmful

  1. So we decide to save the half-day or the whole day that the team might spend discussing design….
    Gee, let’s save time on our project by not designing it! That will help!

    One of the implications of your statement (that you clearly don’t understand) is that one of the major outcomes of task estimation is design. It’s going to happen at some point, or your quality will suffer.

  2. Dear Somebody,

    If you think design only happens up-front, even if only at the beginning of an iteration (say during estimation), then you’re simply missing out on one of the key benefits of the agile approach – evolutionary design.

    Evolutionary design allows the system architecture/design to evolve organically – through feedback from the code itself, testability, build-times, maintainability, extensibility, and customers seeing the thing in action.

    Evolutionary design is possible because of the coming together of all the practices of agile (e.g. XP practices) – unit testing, continous integration through automated builds, constant refactoring. These things make it possible to make “design” a continuous process, indeed it is done just-in-time as and when needed (all the time). This eliminates the muda of the “upfront design” that gets thrown out the moment someone writes the first bunch of code, anyway.

    The possibility of this more practical approach of design makes estimation a somewhat orthogonal issue, and hence the point of my post.

  3. Amit:
    If you don’t do ANY design up front, as you suggest, you’re going to be in trouble. Everybody will go off and do their own thing, solving the same problem different ways. Then when it comes time to integrate part A with thing B, you get redesign what you started out with.

    If you think that your team can just go out and start coding something without ANY design up front, then please tell me. That way I can avoid your products/sites/etc.

    PS: nowhere did I suggest that design happens only up-front. I only suggest that some design must occur up-front. Your comment is just a broad, hand-waving debunking of waterfall. I don’t like waterfall, and make no suggestion that it’s the right way to do anything.

  4. Dear Somebody,

    No, it is not a debunking of waterfall, because the post was written after more than 5 years of doing agile, and was based entirely on real-life experience. The post concerns itself with agile methods themselves, and a potential way to gain even more efficiency by removing some waste by making the process leaner.

    Having said that, I don’t believe we’re in complete disagreement. I understand the need for some upfront design, of course. I’m only saying that this happens on a daily basis – before anyone makes a significant design decision, they talk with the rest of the team (or at least with those team-members that are most concerned with that part of the code-base). I also believe that strict design documents are often of very little use, and they should be skipped in favor of quicker working software, refactoring based on feedback (from the code, from BAs and QAs, from users, and the dev-team), tests (unit and functional), continuous integration, and collective code ownership.

    These practices make it possible to avoid having to do the less than useful traditional up-front design, while still ensuring that the system evolves a coherent and good design, without inconsistencies. There’s a lot of literature around how this works, and about these practices as well.

    Hope this clarifies my earlier comment. Also, my original post wasn’t about design, but about spending time on detailed estimates – whether or not the team does up-front design.

  5. Certain amount of design has to be done upfront. Defining the foundation or the core arhitectural patterns of an application is done best before any coding starts. It’s not easy to quantify the optimum amount of upfront design. It’s more art than science.

  6. Amit,

    I think you gave a balanced opinion at the end or your post, but for the first 90% of it you completely ragged iterations. I think that was unfair if you are going to conclude that only advanced teams should avoid iterations!

    In addition this posts concentrates on how iteration level estimates are evil(except at the end where is implies most agile teams should use them), but I think that argument only holds up if you say iterations are evil, which you also allude to only at the end. This post would have been more compelly, I believe, if you would have started with either Iterations are evil, or Iterations and iteration estimates are evil, but only for advanced teams.

  7. I love this! It’s great to see this idea of stopping estimating catching on. Have you seen this post of mine from two and half years ago? Stop Estimating and my explanation of where there really is value, Agile Estimating. I really feel you’ve come to the same conclusions independently. It’s great to see this corroborating anecdotal evidence to back the idea that estimation is waste. Thanks. David

  8. 2 responses:

    –First–
    After 30+ sprints, we have recently stopped doing iterations. It makes us much more re-active (in a good way), and it is a temporary decision made for that reason.

    We have some hard deadlines, as well as the need to respond quickly to some short-term changes. The team seems to be successfully self-organizing around this, with some people taking on the task of planning and tasking longer term items, and some being in a more re-active mode.

    The lack of iteration structure does have a negative (stressful) impact on some team members.

    We are still trying to break things down to at least the “story point” level, i.e. items that can be completed in less than a week. It is a challenge for individuals to task new items, because they are so used to having myself drive those work breakdowns. I am assisting as necessary. Perhaps in a pair-programming environment (we are not), this would be easier.

    –Second–
    Part of agile development is building up a trusting relationship with a client. Much of that trust is based on being able to deliver what we promise. In fact, in my experience it was a major selling point for Agile – we’ll be able to deliver better estimation, without the large cost of BDUF.

    It seems to me that the vague promise of “we will have some working software for you on date X but we cannot say whether it will meet your high priority goals” may be insufficient to sell this to most clients.

    So this can probably only work once you already have a good working relationship with a client.

  9. Interesting article, definitely got me thinking. I’m not sure I’m ready to abandon them completely, but I’ve had a different experience – our estimation meetings are usually quick, and the team estimates are reasonably accurate.

    A final thought: the thing that stuck out to me was the statement about long and not very useful planning meetings. If you have meetings like this, then the meeting facilitator/scrum master/project manager isn’t doing their job! I would wager that time spent in unproductive meetings in general is a source for more waste than the time developers spend doing detailed estimates. This includes scrum/agile meetings, pointless team meetings, brown bags, all hands, etc.

  10. Amit,

    Let me firstr say, I am sorry that I did not read this blog shortly after it was written as I expect my comments to have few reviews.

    That said, I have been involved someway in developing software for over 20 years, mostly at what is considered the programming or development level.

    I have read your blog about not needing iterations, as well as, estimating as evil. Since most people are passionate about their position on the Agile Philsophy I have usually not commented.

    However, this blog in my view is leading, at least those that practice the Scrum method, down the “garden path”, well at least the wrong path.

    So while this blog, and having read some of your other articles are well written, there are several things that are of concern.

    First it seems, you misunderstand estimating and planning. The estimating is done in this project when you are assigning story points and trying to figure the size of the project.The meeting before each sprint is not named the re-estimation meeting, but the planning meeting. Of course you could be using the word re-estimating just in suppport of your point. In which case, it weakens the argument, since its use is incorrect.

    Another point you talk about in the so called re-estimation is using real tine. In our Scrum practice we use real time all the time, so I could ask if you want to stop estimating then what good are story points, but I digress.

    Your blog pointed out that you still ask the developers to break the stories down into tasks, when they do this, isn’t it planning, and from my expeience with our team the time they say it takes for a task is pretty accurate, unless some impediment happens.

    If after this planning meeting for the tasks to be completed for the sprint, the developers do as you put “All that stuff changes the minute someone writes the first bunch of code” then something is very wrong. As the old adge says “when you fail to plan, you plan to fail.”

    Of course you say there is still planning and call it evloutionary desgin. In the need for this part I am in agreement with you.

    Now for communication sake, let’s spend amoment considering the busines side stakeholders. Since your blog had little in consideration of thhese folks. Given that all that stuff changes how do they know what is going on? I’ll bet the team will spend more than that 10% that you say is wasted trying to inform the business side what is going on. Oh yes, you will be causing more work and lost time for them too, as they have to make more adjustments. I’m sure they will thank you for that. It all works together.

    Since you have taken the approach to be a bit inflamatory, which I must admit was quite succesful, I am firing back with some of my own.

    It seem like you and Mary P (who I consider a great writer) along with others just don’t get it. You want to start talking about “lean” from manufacturing or construction techniques.

    Im my view even the great Hopkins, founder of much of CMMI. doesn’t get it.

    My view comes form a place where I worked where we carefully evalutaed CMMI and ISO 9002 for over a year to see which would be better suited..

    CMMI and lean are to much like manufacturing physical (hardware) things. So enough already, creating Software is more like wrting a report or developing a proposal. Business people have no problem doing a draft and then re-working whaterver it is a number of times.

    I think it would behoove us all to try and find ways to make “Software” faster to develop, higher quality, like the though leaders Stherland, Schwaber, and Cohn have done.

    Len Whitmore, BA

  11. Great post. It’s really good to see hard questioning around the idea of ‘why we’re so keen on estimating’ in the industry, even if I don’t completely agree (I suspect overall neither do you, completely) with the premise that ‘all estimating is evil and unnecessary’. Some semi-random thoughts around that area (that, taken as a whole and thematically, might strike **someone** as a cogent and cohesive reply post – who knows?)
    — ‘The need for accurate estimates’ is one of those deeply-grained assumptions that no-one ever seems to question. People can give textbook answers as to why, but when you compare reality to the theoretical reasons, they don’t jibe
    — One answer to ‘why we need estimates’ (an answer implicit in the literature of ‘old-school’ software development as well as agile) is ‘to help the business plan’ (timing, budget, etc.). However, in my experience, time and again, both in internal IT shops and ‘external product development shops’, it begins with the pretending that we can estimate things accurately, and then when the dates aren’t hit, what adjustment is made? just ‘get it done as quickly now as possible’. No ‘we’ll drop this feature’ or ‘if we exceed the budget by this much the project isn’t worthwhile’ or any of that stuff touted in the older textbooks or the new Agile textbooks. Just ‘get it done as quickly now as possible’. You alluded to this in your post (i.e., ‘what is necessary will get done when it gets done, and we just have to live with that’). If the decisions were really being made based on accuracy of the estimates, I would expect to see some changes when the estimates change (in more Agile / RAD / whatever shops, the changed estimates are earlier. In more traditional shops, the changed estimates are often 11th-hour. But in any case, in my experience the result is usually the same – ‘just get it done as fast as you can’)
    — I’ll go ahead and say it: deep down, most managers, business sponsors, product owners, all those types, cling to ‘accurate estimates’ for one main reason: they have a voice of dread that tells them they don’t have any control over a software development project (who does, when it comes right down to it?), and insisting on ‘sticking to estimates’ is their ritual, their talisman, that gives them the illusion of control and is the only thing that can quiet that voice
    — I think the reason a lot of Agile books don’t just come out and say these things outright is because then the people who need to buy off on Agile would not buy off on it. I can understand this – in my opinion, Agile is overall a Good Thing, and why encourage ‘throwing the baby out with the bathwater’?

  12. Awesome article! I just found this and coincidentally just wrote a much shorter article for the Scrum Alliance website that is in total agreement. I have also noticed the connection with the Heisenberg Uncertainty Principle. Thanks for that part of the discussion.

  13. A lighter brighter and whiter skin by using a flaw less gleam will be the dream you hide in your eyes every night while jostling down using a duvet to comfort your slumber. However when you awaken many of the hopeful expectations with the night glided by diminish with the rising sun. Aging is one of the conditions of life which cannot be defied and you is effective in reducing its effects quire considerably on your skin. By using Perfect Radiance age defense serum it is possible to quash off signs and symptoms of aging which can make you search dreadfully creased and scarred.

    outstanding Pefect Radiance

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s