Lean software development: An iteration-less agile process

A lean software process offers a lot of possibilities. In this post, I want to try and explain why I think iterations are superfluous in a good agile team environment.

Premises

I thought I’d present my reasoning as a set of statements that together build up the argument. I’ll try to address the following things – business analysis, estimation and tasking, ensuring communication, iterations, requirements prioritization, planning, showcasing and closure, taking breaks. There’s probably more, but these are the areas where most people ask questions about this approach.

So here goes!

Analysis

– Business conditions and requirements can change quickly.
– Analysis therefore, is only valid for as long as the relevant business requirement is valid.
– To ensure that the team takes the latest requirements and information into account, therefore, it makes sense to delay analysis to keep it as close to development as possible.
– In an ideal world, analysis could be done real-time. Or, as they say – just-in-time.

Tasking (work-breakdown)

– Tasking a user-story is valuable.
– Developers don’t really pay attention when other developers are tasking other stories. Especially in large meetings. They communicate and figure things out in other ways.
– Developers often don’t even remember how/why they tasked something a few days ago. Sometimes, things change enough to make their earlier tasking less valuable as well.
– It makes sense, therefore, to just have the pair that’s going to develop a card, task it.
– It is also more valuable, by the same reasoning, to try to minimize the delay between tasking and implementing.
– The ideal is right before implementation – in other words – in real-time, or as they say – just-in-time.

Communication

– Large meetings are inefficient.
– It is valuable to have developers immersed in the domain.
– They learn from each other, and from BAs and QAs during discussions. However, the value of this exchange is diminished when it occurs in large meetings.
– The best form of communication is organic – face to face, across the team-room, during work, a little now and a little later, as required.
– Domain related (and other type) information , therefore, is not distributed only during IPMs, nor is it the most desirable form of this exchange.
– The best way is to have domain people and developers co-locate and encourage conversations and daily interaction

Iterations

– The duration of an iteration is arbitrary.
– The only important factor is that it shouldn’t be too long. People have discovered that 1 to 4 weeks work well. Can we do better?
– Iterations are an artificial construct, and just like estimation, they don’t (and can’t) alter the amount of inherent work.
– Zero is the smallest natural number.

Prioritization

– Customers/product-owners are constantly learning new things that impact requirements.
– Prioritization of stories must, therefore, be an on-going activity. It has nothing to do with ongoing development iterations.
– Even a developed user-story (the partially-developed product) provides feedback into this changing understanding and priorities.
– It therefore makes sense to reduce the cycle-time of developing a story – to allow for maximum, and fastest feedback.
– Prioritizing stories can, and should, therefore run in parallel, at all times.

Closure

– Showing progress to stake-holders is important.
– A work-in-progress should be show-cased to them whenever significant features are implemented.
– This implies a non-deterministic schedule. Pre-planned showcase-schedules therefore are not optimal.

Breaks

– A hard-working team deserves to take breaks and have fun.
– This again should be on an on-demand schedule – especially after a burst of extra-hard-work or overtime.
– This also implies a non-deterministic schedule.
– This holds good for things like retrospectives as well.

Recap

– Prioritization is an on-going, parallel activity that runs throughout the duration of the project.
– Analysis ought to be done just-in-time.
– Estimation adds little direct value to implementation.
– Tasking ought to be done just-in-time.
– An iteration is an artificial construct, and shorter the better.
– Showcasing work is important, and taking breaks is important, and these are non-deterministic activities and shoe-horning them into a ‘regular’ schedule is sub-optimal.

Proposal

– Reduce iteration lengths to zero. Do away with the artificial construct altogether.
– Ensure business analysts work with product-owners and stake-holders to maintain a constantly prioritized list of user-stories.
– Ensure maximum effectiveness of the development team by allowing business analysis, and tasking, and development to occur in a just-in-time manner.
– Showcasing product, conducting retrospectives, and taking breaks should occur whenever needed!

Final pitch

Hopefully, I’ve been a little more coherent in my thinking here than I normally am during my… well, incoherent rants. Ultimately, I want to be able to remove all parts of a process that adds no value by itself. Any opposition to this proposal would have to be in terms of what value iterations provide, and that this process will be unable to.

Oh, and also – this is a work in progress 🙂

Extra credit

– Do away with re-estimation altogether.

Coming soon

– What requirements should be satisfied in order for a team to be successful at iteration-less agile?
– What impact does this have on a client-vendor relationship: project management, expectations-management, risk-management and so on?
– Does iteration-less agile make life difficult for development teams that like to have short-term targets? E.g.- ‘200 points this week’?
– Possible disadvantages of saying no to iterations (everyone else is doing it!).
– Also – applying more lean principles to traditional Agile processes.

** Accurate estimate: this phrase is quite an oxymoron.

24 thoughts on “Lean software development: An iteration-less agile process

  1. I really want to see where you’re going with this.

    I actually came to a very similar conclusion when I was at a Lean Software Development seminar with NetObjectives last week. We sort of left it as “iterations are practical artifacts for various reasons, but they are not actually implied by the lean principles”.

    I can see some value that iterations provide (largely economy of scale in deployment and other overhead things) but there is something really seductive about the idea of just doing things in a totally incremental way.

  2. We recently switched to using iterations from an iteration-less continuous cycle. I’ve had very similar thoughts. I’m also curious to see responses and discussions on this topic.

  3. You’re referring to “iterations” as the arbitrary cycles used in planning and estimating (like “financial year” or “calendar month”). But you must still go through many feedback cycles to reach an optimum solution you still have to iterate. Is your beef with iterations as a project management construct?

  4. Jason,

    That is correct. I agree one needs to iterate to reach an optimum solution – my point is that those iterations can be fused into a continuous process of do-feedback-do-feedback… making the ‘iteration’ zero time-units long.

    So yes, my beef would be with the construct of the iteration, which I believe is unnecessary, and by removing it, one can make the process leaner, which gives it all the benefits associated with a lean, JIT approach.

  5. Nice and interesting post. I agree with a lot of lean principles from a customer centric view. But I would be more interested in learning how to handle process abuse, since lean and agile , are quite easily prone to abuse internally.

    IMHO , As long as you have a set of smart,focussed and committed team of people, you can take risks of going leaner. But with a mediocre (not only in technical aspects) team this would be difficult.

  6. One advantage of fixed sized iterations is that they provide a rhythm to the project. This helps give external stakeholders a sense of predictability and lets team member get a sense of achievement.

    Another advantage is freezing the requirements which gives developers a sense of stability. Sure you can’t really freeze the requirements for a long time – which is why the iterations are short. but at least they are frozen for that time (if something really major happens you can always cancel the iteration)

    It is true that estimation is not accurate- the larger the task are the more skewed the estimation will be. However if the estimation is for a task that is a day or less – most of the developers I’ve seen provide a decent estimation. Also estimating relative size is easier

    This doesn’t mean that iterations need to be waterfall-ish- quite the opposite – for instance, as you mentioned analysis is better if it is “just in-time” etc.

    Arnon

  7. This model works with service-oriented teams. (Think IT help-desk, or a group that makes reports.) The trick for software development teams is being able to define functionality in small enough increments to go from not-checked-in to “done” in one check-in.

  8. do-feedback, do-feedback, do-feedback…

    Sounds like iterations to me. You get some information, stories or whatever you want to call them, and work to implement just enough that the customer can see and understand what you’ve done so far. Then you show it to the customer, get feedback (more information–revisions to the stories, new stories, etc.), give feedback(new estimation of velocity, etc.), and implement again. How frequently? As frequently as possible–the rule has always been to prefer the shorter time frame. However, the time interval CAN’T be zero unless you happen to be pair programming with your customer, getting “real-time” feedback (even then it wouldn’t REALLY be zero, but pretty close). The time frame is going to depend on the availability of your customer–how much time he/she can devote to giving you feedback and how often. In a good environment, this will be very frequent, but it’s never going to be 100%.

    I think what’s really needed is intelligent, dynamic iterations:

    Development Team: “Stories A, B, and C could each be finished by a pair in a week. Stories D, E, and F would each take a pair two weeks.”
    Customer: “OK, quicker feedback is better, so lets do A, B, and C” OR “It’s more important for me to see how D, E, and F work, so let’s do those”

    Development Team: “Story F, which we thought would take a pair two weeks, actually took one. At our last meeting, we elected not to work on story A, which should take a week. Would you like to meet, or have us start on story A?”
    Customer: “Let’s meet so I can evaluate Story F.” OR “I can’t meet until next week. Go ahead and do A.”

    So, depending on the circumstances, the iteration might be either two weeks or one week. Generally, the velocity for a one-week iteration would probably be around half that of a two-week, but you’d have to observe and tweak.

  9. Hmm. I think timeboxed incremental software develoment has been pretty well proven to work. I’m not sure about your definition of an iteration but I understand it as a timebox. If you take away iterations and try to do everything just-in-time, you end up with chaos.

    It seems that the “ideal view” of software development is going from one extreme to another. First there was chaos and code-and-fix, then we moved to ultra tight inflexible waterfall, now the trend seems to go towards extreme IID or even chaotic hacking again. The key is to tailor the process to match the requirements of the project at hand and do _enough_ upfront planning, design, analysis and prototyping – no more, no less.

  10. I see your point of view – I do want to add though, that most people thought (many still do!) that agile doesn’t work very well, and that not doing design and planning upfront and instead doing something like Scrum is very chaotic. The leap from waterfall to agile was just as much a step up (and a change in thinking) as a step from agile to Lean will be.

    More to your point, also, is my view that if you >i>do improve your process to truly enable just-in-time, then it will not seem chaotic at all, instead, it will be a highly efficient flow of work-product (features, fixes, etc.) from conception to customers. I’ve written some more about this here – why reduce cycle-time – towards iteration-less agile.

  11. I think too much of your argument rests on the unproven premise that estimation is unnecessary. If you accept that from the business perspective it *IS* necessary, then iterations are required to provide points of estimation as well as to confirm that work is on-target.

  12. I totally agree with Arnon that a key benefit of a short duration sprint is fixed scope so that the development team can FOCUS. As he pointed out, you can always cancel a sprint if the scope has change dramatically and if you want quick feedback, have shorter sprint cycles.

  13. I think that both Arnon and Vishal have good points.

    The thing to keep in mind is that though rhythm can be achieved through the use of fixed-sized iterations – it can also be achieved without them. Just ask a production-support team that is in full swing! For a team that needs the formal closure and validation of an iteration close and subsequent kick-off of the next one, by all means – iterations are a great idea.

    When a team graduates to realizing the true essence of what happens within those iterations, it realizes they don’t really need the artificial construct at all.

    In the end, agile methods are meant to be adapted to the situation at hand. Lean thinking, as a management philosophy, helps focus on those things that increase value. It’s application affects those things that are considered value, and by corollary, those that aren’t. So, in the case where the ceremony of iterations is no longer considered a net addition, one can do away with them to achieve an even higher throughput.

    Nathan, above, for instance talks about a situation where his team no longer felt that the idea of *fixed* iterations was useful – so they moved to intelligent, dynamic iterations. What might be the next step for such a team?

    As far as predictability is considered, you can achieve that whether or not you have the construct of iterations around batches of work. After all, Toyota, the ultimate lean enterprise, can predict with (more!) accuracy when a car might get to market! After all, in software development, velocity is a perfectly good measure of speed, and you don’t need a particular (or fixed) denominator to calculate it (because it is simply a ratio – amount of work to time).

    Finally, as far as focus is concerned, the team should always have focus on what it is doing. Iterations, by themselves, don’t address this. I’ve seen many teams that needed help because their stakeholders were setting unfocused, even conflicting goals within sprints. As long as you are able to ensure that the highest priority items are being pulled off the backlog at all times, you should be fine. (Again – priority can and should be defined in terms that make sense for the project and situation).

  14. If I remember correctly from high school geometry, one is the smallest natural number. In another of your posts, I believe you mentioned going to an iteration size of one, which would certainly make sense. Is this what you meant?

  15. I guess I do. Basically, what I’m saying is that the typical time-based window called Iterations are an artificial construct, you don’t strictly need them to be successful. You can basically treat each feature – taking it from concept to deployment – as an ‘iteration’. This is the idea of an ‘iteration-less’ agile approach.

  16. […] Lean software development: An iteration-less agile process « epistemologic – "I thought I’d present my reasoning as a set of statements that together build up the argument. I’ll try to address the following things – business analysis, estimation and tasking, ensuring communication, iterations, requirements prioritization, planning, showcasing and closure, taking breaks. There’s probably more, but these are the areas where most people ask questions about this approach." […]

  17. I like it. Source control tools used in the correct way could empower a development team to do all this while still providing a releasable set of features at any given point of time. Perfect!

  18. […] How does this relate to planning and tracking? Because those teams that truly understand what being agile and lean is all about, focus on the activity of planning, and then react responsibly to changes that follow. Iterations are used to gather more and more knowledge about the system being built (while delivering to production at all times) so that better decisions can be made and better priorities can be set. Really mature teams don’t even need iterations. […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s