Estimation considered harmful

Or leaner Agile – Part I
(Part II is here)

This is a rather long post on a topic that has sparked many arguments whenever anyone – colleagues or friends – brought it up. It is about applying a key concept of Lean to, what some may consider, a rather extreme degree. And I’ll admit this – the title is a little misleading, and deliberately a bit inflammatory. Anyway, here goes…

The what

I want you, dear readers, to walk with me. Walk with me and follow along as I travel the path of an imaginary project that we, as forward thinking Web 2.0 visionaries, take from an idea stage through to construction. And during this journey, I want to show you what place estimation as a practice ought to occupy in the ideal software development life-cycle.

So, first we need an idea. Hmmm, a quick look around tells us that a mobile social-networking website would be cool – it would be just the thing to get easily funded, and hopefully, bought by Google in short order. Having satisfied ourselves that this is a solid business plan (the VCs are happy with it), and that it’s (clearly) bound to work, it is now time to get started on getting the thing built.

Now, what features should the site have? Since we are the founders, we brainstorm among ourselves, and since we’re hip, we even ask a couple of potential users to participate. At the end of the day, and after a long and grueling session of arguing about what’s useful to teenagers and what isn’t, we have a nice, shiny list of stories. Again, since we’re hip, we decide to call it our Master Story List.

The how much

OK, how much money are we going to need? And when can we launch the first release? Certainly, our friendly VCs will ask us such questions. Hmmm… time to call in the cavalry. Through our connections, we hire the best developers we can find. We also hire a couple of QA folks. Since we plan to play the role of the business-analysts ourselves, we have a full quorum. We’re ready to do some estimation. We don’t really know too much about the system, at least none of the specific details, and nor do we know exactly how certain things are going to be implemented. What we do know, is at a high-level – what we want built and what each feature might look like. With this input, and with the years of software development experience that the development team has, we assign story-points to each requirement on our Master Story List. After another couple of days of intense sessions (discussing and arguing about what each feature is, how it relates to the market and to other features, and how it might be implemented and so on) we finish our first cut of the sizing exercise. Our story-points have a simple scale (lets ignore unknowns for now) – 10, 20, 40, and 80 – where a 20 was a small story, 40 a medium, and 80 a large. A 10 was assigned to really simple stories. When we added everything up, we had a total of 13,340 story points.

Wheee! We’re done estimating! At least for now. Again, from all the experience that we have as a group, and from our best guess of how things might go on this particular project, we think we can get 600 story points done every couple of weeks. The full project now appears to be about a year in length. We also want to do multiple releases so we can release something quicker. So we pick the features that we think we absolutely need for the initial launch, and it turns out to be about half the overall set. We therefore plan for a six month initial release, followed by several shorter releases. We’re ready to go!

By the way, when we spent those few days locked inside the conference room discussing features and so on, we also derived a list of questions, assumptions, and risks. We’re going to use that list as a starting point for our project-long risk-management activities. We’ve got all of it written down (in a nice Excel spreadsheet, no less) and the plan is to update and monitor everything on a weekly basis.

Go! Wait! Iteration planning

So – we’re ready to start! We bite off a bunch of work that we think we can get done in the first couple of weeks – and the developers go at it. How exciting! Now, we’ve read books on Agile software development, and some of them recommend that at the iteration planning session, we should re-estimate those stories that are planned for that iteration. They say that this detailed, more granular, task-level estimation is a key part of the iteration planning meeting (IPM), or sprint planning meeting. Much of the literature also says that we do this second level of estimation in real units of time (as opposed to the story-points we used earlier) – so that we can plan iterations better, and track progress better.

We think about this – and question the value of this effort. We already know what needs to be built – for this sprint and for the release. More accurate estimates will not change the amount of work, much less reduce it. So we decide to save the half-day or the whole day that the team might spend discussing design and implementation details to get at the more accurate estimates (wow, that’s some oxymoron, huh?) All that stuff changes the minute someone writes the first bunch of code, anyway. So we decide to let the code, and hence the software, speak for itself. We let the developers do what they do best – we know they’ll use good software development practices to ensure high quality. The QA folks are here to test the application as well. The actual working system is what matters, anyway!

We do, however, ask the development team to break things down into manageable chunks of work – in other words, they task each story out. This is useful to them, because it helps them think through things, and keeps them focused throughout the implementation of each story.

Keeping it simple

In making the decision of not bothering with more detailed estimates, we also thought of this – there are only two things that we can play with – scope, and schedule. Quality has to be high – that’s non-negotiable. We’ve hired good people – they’re enthusiastic about the project, and they care about their craft. We’re sitting in the room with them, and helping move things along whenever they’re blocked on things. If we want to get done quicker, the only option we have is to pull stuff out of the release plan. If we want all the stuff that we had decided on, no matter if things take longer than we hoped – the only option we have is to delay the release. It’s that simple.

Remember the old days, before this whole Agile thing, when people thought that the way to control the complexity of building software was to freeze requirements before starting work? That didn’t work so well, did it? Today, another delusion persists in the minds of many managers – that of being able to predict the date of completion with accuracy. Just like the weather (the most powerful supercomputers can’t predict weather with any accuracy more than a few days out, and they often get it wrong anyway), software is too complex to think that by controlling a couple of variables, one can control the trend-line. Even that control doesn’t have predictable results – by de-scoping a complex story (whatever complex might mean to you), is there a quantifiable way to say how much time was saved? Or by re-factoring a gnarly area of code, can you tell how much efficiency was gained?

Why do we still persist in trying to adjust for velocity, drag, complexity, performance, estimates, re-estimates, actuals, rate of scope increase, etc.? Why do we expect that by running massaged numbers we can produce the date? After all, what software team ever announced a date that they actually met? These days, many companies just stick the number of the year at the end of the name of their product – Office 2007, Windows 2003, Pocket PC 2006. They’ve given up the charade – they’re now saying they’ll just deliver it sometime in the year. Just keep your fingers crossed.

So now what?

So then the question becomes, if the software development is going “slow” (in other words, estimates turned out to be just that – estimates), when will we be done? And can we do anything to speed things up? The answer to when comes from the story-points we’d assigned to every story coupled with the data we gathered over the past several iterations. If it appears we’ve been getting done only 450 points each time, then, hmm… we’re going about 25% slower than originally “planned” and realistically, our six month release is looking more like an eight month release. Again, fairly simple math gave us the answer. The second question is also fairly simple – and in fact, it has nothing to do with estimation at all. It is a quintessential question about software development – how can we deliver faster? We can’t answer that – maybe no one can. Doing things right, hiring smart people, these things help. For now, we either accept a delivery in about 8 months time, or we trim scope.

The thing to note, also, is that if we’d spent the time to do detailed estimation (at the task level for each story in the sprint) – we’d have lost another half-day to a day every iteration, and we’d still have no real change in the outcome. Sure we’d have data as to how accurate (or entirely off) our development team was with their estimates, but that would be about it… it wouldn’t change the amount of time they’d have taken. And we’d be short about 5 to 10 percent of the time to actually build the software. Oh boy, we sure made a good decision of not going even slower!

Some books on Agile also talk about how important the planning meeting is… what with the estimation effort, and the communication that goes on during it. If you’ve been following along, we just thought through why detailed estimation is a wasted effort. Now lets talk about the communication aspect.. The theory is that this meeting allows developers to find out what other developers are working on, and also gain understanding about the other areas of functionality. All this as the business analysts drone on about how this feature or that. Didn’t we catch sight of those two QA people dozing off? And that developer? Some of the other BAs looked pretty bored too… the reality is that long meetings are not good forums for anything, much less exchanging information about something as complex as software. When developers are curious about someone else’s code, they bother each other, and talk about it at a whiteboard – our whiteboards are always filled with the remains of all these technical discussions… squiggles that only mean something to the techies.

Re-estimation

The literature also points us to another advantage of detailed estimation and re-estimation. The idea is that if we keep careful notes (and numbers) about what and how the team originally estimated each type of story, then if certain estimates were consistently off, we could go back and fix those types of estimates across the project. Thereby, we’d improve the accuracy of the end date quickly and easily. This makes several assumptions – the biggest one is that the original estimates were all consistently inconsistent. Ouch, now we’re in the land of mirrors. Not only were they consistently off, but they were consistently off for the same reason – a leap of faith, really, given all the variables.

Now, I’ll admit, I don’t see any reason not to try and “fix” incorrect estimates, especially after you know more about the system. It’s just that you don’t need the whole detailed estimation waste-of-time for this (or actuals for that matter – they’re evil also and I’ll talk about them another day). Any good development team will realize that certain estimates were made on an assumption that was wrong (or whatever), and will provide information that will enable management to go back and correct for such things. This is just a normal and an everyday software development activity – you make an assumption and move ahead, when things change, you go back and adjust. Normal communication habits are all that one needs. Some people say developers are not capable of doing this – they only care about writing code. They say they’re geeks and don’t know how to communicate, and don’t know what’s important to management. I’ve even heard that you can’t trust developers to raise visibility of such things cause it’s like admitting a mistake or something. Oh boy – if you have such developers, your project is doomed no matter how good the estimates are.

However, like I said, there is value in going back and correcting your plan based on more information or a risk being realized. Again, this should happen automatically, every day! This is also a happy side-effect of the regular risk-management sessions we talked about earlier. Finally, just like quality, risk-management is everyone’s problem.

Further planning

One more question that invariably gets asked when this topic is being discussed: OK, so if I go with this approach, then how do I know how much functionality to plan for in a given sprint or iteration? I have two answers – the first is to simply use yesterday’s weather – if you got done 500 points last sprint, try the same thing this time. If half your team is going to be gone on vacation, try shooting for 250 points. Again, simple and as effective as anything else.

The other answer is that iterations are evil, they add no value, they’re considered harmful, and should not be done. But that’s fodder for another post.

Anyway…

So this is my pitch about why detailed (or task-based) estimation is in general a waste of time and resources, and is a contributing factor of a lowered velocity (the 5 to 10 percent we talked about, depending on how much time is spent every iteration, more thanks to the interrupted flow) . It also adds an unexciting element to the software development process that adds no value to the actual deliverable.

So, do I believe that all teams ought to abandon low-level (or detailed, or task-level) estimates? No – I believe this is an advanced technique – and can only be applied effectively if all the players in the development team truly grok software development. It assumes that all players are equally enthused by the art of building software, and are engaged in the iterative process of development that is Agile. If your team is just starting, or you have team members that aren’t experienced using Agile techniques, and especially if you have a project manager that doesn’t fundamentally understand agile and/or lean software development, then you’re most definitely better off doing things the traditional way – and estimating at the task-level.

For others, here’s a way to trim some muda off your process. Flame away.
(Part II is here)

Story points – handling unknowns

I’m a big fan of usingstorypoints to estimate effort on software development projects. It is a good way of keeping things simple and it ensures that teams can be efficient and quick about their estimation tasks.

One question that often comes up is how a team ought to handle stories (or epics) that they just don’t know enough about at a given moment. My answer depended on the kind of scale the team was using – if they were using T-Shirt sizing, then the scale ought to have an Unknown Size (translating to a scale of say – XS, S, M, L, and UNKNOWN).

If they’re using a numbered scale, then I suggest a twist on the UNKNOWN level. I’ve recently switched to using a geometric scale (as opposed to a Fibonacci scale) – and to that I’ve added an UNKNOWN level of 1,000,000 points. So now, my scale is – 10, 20, 40, 80, 1000000.

What this does is it radiates a little bit more information about the total estimate for the project. Lets say you have 200 stories, of which 10 are unknown. Lets also say that the rest 190 stories add up to 8450 points. Now, the total (because of the new level for the unknown items) becomes 10,008,450 points. It is still clear that we have about 8450 points of estimated work, but it also makes clear that we have 10 items that are unknowns.

If the unknown level was only twice or thrice or even ten times the size of a Large story, then the total for the project obfuscates how many items are unknown – and gives an impression that is not quite accurate. This large one-million-point level for unknown stories fixes that. And it also effectively broadcasts the idea that a subset of stories just don’t have the same level of detail and clarity as the rest.

Advantages of story-trees

There are several benefits of using story trees, and they’re primarily in two areas –

Backlog (or Master Story List) creation:
Mind maps, which can be used to create the story trees in the first place, make for an excellent tool for facilitating brainstorming sessions within the team
– They’re also a fundamentally intuitive tool for thinking about things, and this in itself make things very easy

During development:
– It is easy for a developer to look at a story title in the context of its tree and understand the workflow it belongs too, and the business context in which it was created
– It is an easy, visual way to see what stories are connected to what other ones, and how the dependencies might turn out. It also helps in managing those dependencies across those stories, because of its visual nature.
– Speaking of how things are related, it is also easy to see areas or pieces of functionality that are impacted by change in a particular area. Not always, but this is useful many times.
– By color coding the nodes and leaves of the tree, say based on status, priority, size (t-shirt sizes or story points) or whatever else makes sense, one can render different view of the story tree, which can communicate different things in a very easy to understand manner
– It is a very useful tool with which to have meaningful conversations with the customers. When asked things like where scope-growth was occurring, or where they might look to trim scope, the tree becomes a useful talking aid. It can be color-coded to show times ranges when certain stories were added, etc.
– Finally, story trees are useful because they always pitch the topic of conversation (say about a set of stories) in the context of the entire graph of requirements and business work-flows. This perspective is often lost, and is extremely useful when having certain kinds of conversations (the important ones).

Requirements management, user stories, mind-maps, and story-trees

For those who’ve been following along, I recently started a new project team that’s working on pretty much a green-field type of a project. I’ve been using mind-maps to kick things off, in various areas of the teams’ initial effort, and have now extended it as a way to build the functional backlogs that the teams will work on.

Essentially, what that means is this – imagine that we, along with an analyst (or a team of analysts) sit down to brainstorm an area of the application. We always start with the high-level ‘project’, lets call it Todos2Go. Just to retain context, lets also go ahead and list out the high-level functional areas of the application. These include modules we already discussed and elaborated, they certainly include new modules we want to include in our system. So in our example, Todos2Go would start with the basic todo-list creation module. It might also have a collaboration module, a search module with advanced search options, a calendar integration piece, an RSS generation module, an email interface module, a tagging and tag-browsing piece, and others. The analyst team doesn’t worry about trying to capture everything at any given point of time, knowing that they will be revisiting these requirements (the mind map) whenever they need to. So at this point, here’s what we have –

todos2go.png

Remember, we’re only capturing functionality from a business requirements perspective right now. Things like config-management, architecture, etc. are somewhat orthogonal areas. They’re captured in a different mind map. Also, we’re not concerned with scheduling right now – as in, when a particular piece might get built. Those decisions will be made based on business priorities, technical dependencies, and developer estimates. So, onward! Lets continue following the analyst team as they keep using the mind map to go further and further into details.

The analyst team picks one module that they’re going to focus on for the moment. They decide that since they’re just starting the project, they might as well pick the one they know best (from previous versions of the software, or from market research) – a basic task-management feature – the Todolists module. So they start to discuss the things the user might want to do, and/or how they would like to break the functionality down for development. Here’s something they might come up with –

Todos2Go-Todolists.png

The idea is simple. Keep breaking things down as you go down a path of functionality. In other words, go ahead and do plain ol’ functional decomposition. Again, remember to view things from a business requirements perspective. This means that ‘design database schema’ is probably not something that ought to appear on this mind map. The numbers next to the items are simply ID numbers, for tracking purposes, and will be of use later on. This view is what I’ve started to call Story Trees, thanks to a colleague – Roger Marlow.

Also, since most people use Excel to track projects (so do I, and I will continue to do so until Mingle launches), here’s how I capture this meeting artifact (it’s usually a whiteboard thing) in a spreadsheet –

todos2go-storylist-excel.png

The Parent ID column is simple. It is just the ID of the bubble in the mind map that is a ‘parent’ of a given bubble. This way, a simple script can be written to reconstruct the mind map if needed. The reconstructed version of the mind map can sport color-coding based on status, priority, or whatever else one might consider useful. More on this script (or my version of it) in another blog post.

So, I do hope this was useful. This is a simple meeting technique for project managers to use to facilitate requirement gathering workshops or business analysis sessions. Again, as I mentioned in one of my earlier posts, I do not use software/laptop/projector in my meetings because it is just not a good, interactive way to run a meeting. I use a whiteboard, flip-charts, and Post-Its. Later, I use software to capture it.

Coming soon – more benefits of Story Trees.

P.S. – Please pardon and ignore the horrid use of the generic term ‘user’ in my story list. We typically use personas, the focus of this post however is the story trees.

Lean Software Development and the Theory of Constraints

I’ve been trying to apply lean software development principles on my project for a little over a year now. So, when I recently re-read Mary Poppendieck’s Lean Software Development – An Agile Toolkit, I thoroughly enjoyed it and found I was nodding to myself a lot. I highly recommend reading this book because it explains in very simple terms what this whole lean process thing is about, and how it relates to software development. A knowledge of why Agile processes work and what they’re aimed at would help, but is not essential.

For those who don’t have the patience (or time) to read the full book – here’s my version of a summary –

– Strive to eliminate waste. Waste comes in many forms in the world of software development, all of these is waste – partially done work, extra processes that add no value, extra features that have little impact on business value, task switching because of multiple objectives or other reasons, waiting for something, having to go somewhere (to meet someone or fetch something) to get information, defects, and ‘management’ activities.

– Use a technique called value-stream mapping to understand where the process you’re using delivers value and (more importantly) to determine where it adds none.

– Amplify learning through feedback. This is easily done through short iterations and negotiating scope for each of them.

– Think about set-based development techniques.

– Delay commitment – decide as late as possible. At the same time, think about concurrent development.

– Deliver as fast as possible – this enhances learning, and delivers value at the same time. Use concepts from pull-systems and queuing theory.

– Understand the cost of delay – use a product model. Have an accountant on your team who can help with this, and make trade-off decisions in light of what the model says.

– Empower the team!

– Think about integrity – both conceptual integrity, and perceived integrity.

– Testing – TDD, automated functional testing, continuous integration – is key. Refactoring is key.

– Always optimize at a level higher than the perceived issue. Employ systems thinking.

– Embrace the idea of Customer Collaboration vs. Contract Negotiation.

I’ve found that as I started to apply lean software development methods, our processes started to become more and more natural, and that the team became more and more effective. As expected, the team members themselves began to look for new and innovative ways to optimize their workflow, and my job as a project manager became easier and easier from that perspective.

Speaking of optimization, I recall being introduced to the writings of Eli Goldratt by a really smart project manager I worked with in the past – Matt Gelbwaks. Goldratt writes about the Theory of Constraints – his books The Goal and Critical Chain are a really good start to the whole discipline. He has a really simple algorithm to optimize a system, based on throughput accounting

1. IDENTIFY the system’s constraint(s)
2. Describe how to EXPLOIT the system’s constraint(s)
3. SUBORDINATE everything else to the above decision
4. ELEVATE the system’s constraint(s)
5. If in the above process, a constraint has been broken, GO BACK to step 1 but do not allow inertia to cause a system’s constraint.

Applying the Theory of Constraints to improve my teams’ processes, while using lean software development ideas to drive agile practices has proved absolutely invaluable to me over the past year.

I’d be happy to hear from you if you tried similar things on your project… and about how they worked out…

Pay incentives vs. motivation

I’m sure you’ve had many conversations with people about what motivates people, and how to get them to work smarter or harder or to get them to do the right thing. Compensation – money, stock options, bonuses, and other similar perks quite often figure in these discussions. Yet, there is usually a thread about how these don’t really seem to work that well, or perhaps not at all.

Here’s expert commentary from someone who has spent many years studying management practices – including what kinds of incentives work – such that people are intrinsically motivated to do the right thing towards the end goal. This testimony was given to congress, no less.

[Originally from Esther Derby]

Agile: Starting a new project

This one is intentionally going to be somewhat vague. I’m starting a new team this week (we’re using Scrum, and I’m going to be the scrum-master), and I was talking about this with various project-manager/scrummies on the multi-team project that I’m presently on. We were discussing lessons learnt, ideas, tips/suggestions etc. about sprint zero.

I’ve been on Agile projects for several years, Scrum for a couple of years now – both as a developer and as a project manager. I’ve been playing a scrum-master role for the last year – on a large team that I picked up six months after it had been started. Now, it’s time for a new team.

Sprint zero would include setting up the backlog, getting product owners aligned, getting developers who’ve not necessarily used a lot of real Agile practices before educated about things, figuring things out around the architecture, deciding what to track and how, deciding on length of iterations, and so on and so forth. Basically everything that might come to mind when starting a new project.

I’ve got several ideas put together for what might be a good way to go about this – but what are the top few things that come to your mind when you start a team?