A Project Delivery Sanity Test

I’ve been working at ThoughtWorks as a software consultant for many years. During this time, I have done development as well as a project manager work. My most recent exploits were at a large project ($100 million+ program) as the PM on a sub-team of about 40 people.

After working with many clients and projects and also having worked on several side projects during the past few years, I like to think I’ve seen projects that span many characteristics – size, complexity, team composition, geography, domain, and technology. Based on the observations I’ve made over this time I’d like to present a list of things – a checklist if you will – of the most important issues that may determine the ultimate success or failure of a project.

1. A Delivery Focus

Being delivery focused begins with releasing software the moment it is written (almost). Releasing means to get the software into the hands of real, live users. Nothing else counts – developer-done doesn’t count, deploying on QA environments doesn’t count, staging doesn’t count.

Until the software reaches at least a single real user, all the code written (and all the design documents, passing tests, whatever else) is just inventory. (This implies that the goal of making money from the software can’t be achieved, thereby reducing the throughput to zero.)

The biggest benefit of getting code early into production (after creating a value-stream that might actually have some output) is that it brings the kind of focus on doing the right things that nothing else can.

I’d like to make a related point here. One of the few metrics that makes sense on software projects is cycle-time. The team must endeavor to reduce this cycle-time – and should be able to take an idea that needs to be implemented and push it into the hands of users in as little time as possible.

Another important issue of note here is that of usability. Once the software is out there, in the hands of the customers, are they happy using it? Or is it an exercise in frustration? If the users are captive (say internal users of an IT application), then the software still gets used (albeit it is the cause of multiple cuss-words a day). If it is a web-app meant for the general internet population, then the product will tank. Either way, the only way to truly understand and fix usability problems is by releasing it to the actual users.

a) Is your project in production? Are you satisfied with how long something takes to get into the hands of your users?

b) Is the feature-set being designed with a usability focus?

2. Clear Priorities

Speaking of bringing focus to software development, the most important thing is to ensure that you’re building the right thing! This relates to what feature next (and what feature never), what to improve, and what to remove. Setting a priority can not be arbitrary, it must come from a deep understanding of the market, the domain, and from listening feedback from real users.

This is not to say that one should do whatever any fickle user says, but it is important to listen, and include the feedback with the other things that go into prioritizing goals and features.

Without a source for rock-solid priorities, a development team will flounder. Rock-solid means that at any given time, anyone on the team can say what the next few most important things are. It also means that it is rock-solid until it is changed (which can happen at any time, based on changing reality and discovery).

I also like to think of a similarity between priorities and constraints. There can be only one top priority. When there are too many top priorities, there aren’t any.

Do you have a clear set of priorities (at any given point in time)? Does everyone on the team understand them, and how they are set?

3. Stakeholder Involvement

The one thing you can’t outsource is your product-owner function. Many teams seem to forget this, as product-owners spend so little time with the team, that they might as well not be present.

In today’s busy world of everyone playing multiple roles and having too much to do, it is common for people to multitask their days away. The effect of this on developers has been documented and studied – I have seen pretty much the exact same thing happen to busy product-owners.

The reality, however, is that if the bread and butter of an organization is the software their users buy, nothing can be more important than ensuring that it is built right and will garner the customer satisfaction and loyalty that is a must for it to succeed. Nothing.

A side-effect of this is some form of seagull-product-ownership. Here’s an example – the product-owner appears during a planning meeting (or the like), and makes comments that alter priorities (which is OK in itself). This changes what the team has understood so far and sends them into a flurry of activity to try and get things done which don’t necessarily flow with what they were doing until then.

A product-owner must, well, own the product. There can be no substitutes to spending several hours (at least) with the team each week – and truly understanding what the development issues are, and providing the team with critical information that is driving the priorities of the project.

Is your team’s product-owner involved enough with the project?

4. Business Analysis

This item is a larger issue in more complex domains. Remember, to truly complete a story it should be pushed into production. Small, incremental stories that make business sense are thus critical to running things in this manner. To help with analyzing requirements down into suitable chunks of work, they should be created with the INVEST principles in mind.

On teams delving with sufficiently complex domains, having business analysts that both understand the domain and are able to create such stories is a must. Again, the important thing here is the role – and that someone needs to do it. Having a team of poly-skilled people that include this skill-set is enough as well.

For what it’s worth, a good ratio of BA people to developers is about one BA for every one or two pairs of developers.

Does your project run off of well-written user stories? Is it sufficiently staffed up from a BA standpoint?

5. Team Size and Structure

From my experience, this one is a bit counter-intuitive to most people. People seem to think that for large projects (cost, number of requirements, whatever) large teams are required and even justified. There is plenty of research and literature out there that speaks to the negative impact of having too many people on a team. Even empirical observation allows one to reach the same conclusion – after all, how many smooth and successful large projects have you seen?

Ultimately, I think, this issue boils down to one of competent people. I find it somewhat difficult to believe that your average IT shop can attract not a dozen or two, but a couple of hundred developers to a project – and all of them are super-smart, super-motivated, and super-interested in the given domain.

The other issue that hits hard is how the team gets ramped up. Often there are too many people right at the beginning. Teams need to be ramped up slowly at the start – while high-level architecture is solidified (through code, being pushed to production!), and patterns are laid out, things of that sort. As these things happen and everyone understands the code-base well, the team can begin to add new members. This should be done slowly, as each new person needs to reach the same level of deep understanding. Once there are several people who know the existing system well, the ramp up may accelerate a bit.

However, I’m yet to see a project team of about 10 to 20 developers that couldn’t outperform a team of a couple of hundred. After all, you can easily scale up hardware, get more office-space, buy more snacks and food, but you can’t always scale up the rock-star developers that are required for a project to be successful. In other words, it’s probably Ok to hire less of those awesome developers you know if you paid them twice as much as your finance department thinks they’re worth. You’ll end up ahead anyway.

A related issue here is that once a project does get saddled with a large team, how is it sub-divided? Are the divisions made along functional lines (modules) or are they put into fluid workflow teams as the need arises? Is it one large team working on different stories with the whole code-base being collectively owned? Or is it bickering factions that jealously guard their areas of code from any change?

People who have suffered through artificial team separation along ‘module’ lines know the pain this can cause. Throw in the typical office politics into the mix, and the project ends up producing a system that conforms highly to Conway’s Law.

Finally, most people on projects know when these things have become issues – just listen in on any lunch conversation – or for the full details, hang out with the team at the bar.

Is your project over-staffed? Is the team sub-divided along the right lines?

6. Planning and Tracking

I’ve seen many teams ‘do Agile’, while not being agile in any fashion at all. Being agile is about focusing only on what’s important – and ignoring the rest. It means doing just enough of something so that it works as desired and produces the right kind of output. It means not doing anything just for the sake of it, and it means questioning any kind of process-thing which doesn’t add value to the software being produced.

Being agile allows and aids you in being lean.

How does this relate to planning and tracking? Because those teams that truly understand what being agile and lean is all about, focus on the activity of planning, and then react responsibly to changes that follow. Iterations are used to gather more and more knowledge about the system being built (while delivering to production at all times) so that better decisions can be made and better priorities can be set. Really mature teams don’t even need iterations.

What agile isn’t is a set of smaller periods of waterfall-type software development. An iteration planning meeting doesn’t produce a plan that must be met with perfect accuracy. Not meeting an iteration goal doesn’t mean it was a failure. Estimates still need to be treated as estimates, and iteration capacity still shouldn’t be filled to the maximum (basic queueing theory).

Planning on many teams becomes an exercise in craziness. This is especially true when the level of detail that people get into becomes so great, that they might as well just get the work done right then and there. Planning should be high-level goal setting, and as long as each individual team member understands the desired outcome for anything he/she picks up to work on, the intention of the planning meeting has been met.

There is often a tendency to re-estimate work in ‘real hours’, and then to track the progress and completion of each sub-task throughout the length of the iteration. This is a waste of time, and simply doesn’t need to be done. Might as well spend the time on taking the team bowling and spare them the wasteful effort of this sort of planning. At least the team will have a good time, and will get time to gel on a more personal level.

A more insidious issue here is the false sense of security or concern tracking things this way often produces. Graphs, pie-charts, trend-lines – all show the amount of work done, amount outstanding, tasks in progress, how many builds were green, how many bugs were found and how many were fixed, how many hours were worked overall, etc., etc. Instead, how about how many customers liked the product (is it even in their hands yet?), how many releases were made to the customer (boy, are we responsive!), how old a discovered bug is (why isn’t it fixed yet?), how little work-in-progress there is (unreleased software is just inventory), how fast the cycle-time is (turnaround time – from idea to profit) and so on? Metrics do matter, but ensure that they’re the right ones.

Does your project plan at the right level, and track and measure the right things?

7. Technical Architecture

Technical architecture can be a strategic advantage. This is most true if the design of the code allows new features to be added (or bad ones to be taken out) quickly and safely, and also allows maintenance to be done without breaking half of the system. It is an advantage if the turn-around time of a new feature or a fix is so short that the team can run circles around its competitor.

This is a system-wide issue (what isn’t?). If every part of the architecture except one is very well designed, is maintainable and extensible, etc., then that one module will become the constraint of the system when it comes to features in that area. The code-base needs to be kept clean and trimmed of fat at all times, across all areas. (All disclaimers apply – in the end it boils down to the cost of change vs. ROI).

Developer productivity is a related issue here, but is very important. The write_test-write_code-deploy-test-see_results-repeat-checkin cycle is how each developer spends his/her entire day. If the deploy-test-see-results part of this cycle is not down to less than a minute or so (at worst), you can expect major productivity losses from the lack of flow. Build times must be carefully monitored and everything must be done to ensure that developers aren’t waiting around for something like a build result.

Does the technical architecture of your project support a quick response (new feature, bug-fix, enhancement)? Does your project have long build times?

8. Testing, QA practices

Automated tests are the most important part of a code-base. Period.

No one can get a design right the first time, no one can get all the requirements down pat the first time. Things always need to be changed, and software spends four-fifths of its life in maintenance mode. The only thing that can automatically tell a developer that he has broken something is the test-suite. Without this, even making a simple change is a nightmare. Enough said.

QA people are also an important part of this. They are the independent verifiers of the developers’ understanding of the requirements. Testing early and often is the key, as they say, and QA folks need to be part of the development team. They need to be in there while a story is being written up, before it is played, and soon after it leaves the developer’s workstation. And everywhere else with their ad-hoc testing. And they should automate, automate, automate. Why should they do things which the computer is suited for perfectly – repeating a step-by-step procedure and ensure the output is correct? Enough said.

A good ratio of QA to developers is about one QA person for every one or two pairs of developers.

Is your team staffed sufficiently from a QA standpoint? Do you have automated GUI testing in place?

9. Understanding software development

This one is actually the simplest to state, but I’ve noticed that a lot of people have trouble accepting it. Here it is – people who run software projects should know what software development is all about.

If your project is being run by someone who doesn’t really understand the nuances of software development, and heaven forbid thinks he/she does, then you’re in trouble. Generally speaking, of course. I won’t say too much more about this – I just happen to think that people who either write the software or those who manage the people who write the software should know what the heck it’s all about. It just saves a lot of silly mistakes, absurd conversations, rework, and generally helps getting things done right, quicker.

Does your project team have managers that truly understand software development?

So there it is – my first cut at crystalizing many of the issues I’ve seen in the past few years on various projects into a list of nine items. There are other important issues to be watchful for, be sure of that. Also, I don’t think that the issues on any project that is in trouble can be traced to one single point above. It will always be a mix of several things, some listed above, some not.

I hope this is useful though!