There is something to be said about an almost new kind of thinking about software development. Agile folks probably think of this in many forms subconsciously already for it almost forms the basis of the agile philosophy.
This “new” idea is that of business goals – the somewhat quaint idea that any business software that is being developed is to meet some… well, business need. Or goal. Why quaint? Because in the everyday details of requirements, development, QA, deployment, bugs, bug-fixing, maintenance, meetings and what-not, people simply forget what the intent of the system was.
Software is written to help improve business. This is the business goal – make more money by doing something in a more efficient manner or making certain people more productive. There is an important difference between these goals and the actual software that is being developed, and it is useful to keep this in mind.
Why Agile works, is because of this unconscious essence of Goal Driven Development. The idea that people (business people) do not know what they want (quoted as a reason for changing requirements) is not entirely true. They are clear and quite knowledgeable about their business – it is hard to doubt that. What they may not be entirely clear about is merely the software – which is as it should be, because the software really is subservient to the end-goal – that of greater profitability, productivity or efficiency. They don’t care if a certain story card is implemented one way or another as long as it meets their needs – their business needs.
This is fine – business people know their business well and technologists (developers, testers, UI folks, etc.) know technology. I am not implying that techies (used in an umbrella sense of the word – it includes all roles on a software development team) shouldn’t or can’t tell what is good for the business – they can and should. This is because they need to help make the software as good as possible to meet the business goals. What I am saying is that they shouldn’t get caught up in the software itself – so much so, that they forget the intent – the business goals.
From the various consulting gigs I’ve been on, I can say with certainty that in many IT shops, this is precisely what happens. When business people (with the help of business “analysts”) write out requirements – developers stick to these (or at least try to) to the letter. QA folks test against these and again, log bugs unless the software does everything as specified in the requirements spec. UI guys aren’t happy until the interface does things exactly as their plans had laid them out. They all make their activities about the software – what it looks like, what it does very specifically, etc.
They forget about the actual goals. Do all the members on your team know what the business goals of the project you’re working on are? Do they know how it will improve the business of the stakeholders? Do they know what the real goal is? I bet you a dollar, most don’t. They probably don’t even care – since they’re not the business, they’re IT.
This is why Agile works out better. The constant involvement of the customer makes it easier to perform constant checks that the software is going to (or already is) deliver business value. Not just that it is matching up to the spec. This is the reason you don’t need to do complete upfront design – in fact, it explains why you shouldn’t.
The reason is that business folks don’t know about software! And why should they? They don’t care about much more than getting the desired results from the business point of view. It is the reason they have an IT department or the reason they hire highly paid consultants to come in and do the software. They can and do need help with the details of the software and how it should do what it is meant to. And that is really the industry you and I are in – it is our job to improve the usability of the software and leave it maintainable and extensible and all the good stuff – but most of all so that it achieves the goal it was meant to.
Once again, that is why Agile is more successful than other methods of building software. The quick feedback loops with everyone (especially the customer, who doesn’t know much about software) helps in defining and refining the “requirements”. This is the cause of changing requirements – they aren’t “changing”, they are going through the natural and obvious process of discovery. (And most often, changes are really made to the way the software functions and not to the underlying business process and certainly not to achieve and entirely different business goal). This is what IT teams should allow and encourage, for that is their real jobs – not complaining about how requirements are constantly changing under their feet and that they can not get anything done. Any process that attempts to freeze requirements is therefore inherently flawed.
This is what I try to do on project teams – encourage client IT folks to think in the systems-thinking 1 kind of way. It is the bottom-line that matters most, and everyone – the business and IT, needs to work together as a team, to make the company more profitable so that everyone wins. And by the way, the software may help to do that. It is more about the business and less about the system itself. That’s why Agile makes the difference – it helps this kind of thinking, making it possible to create business value faster and of higher quality, with everyone pushing in the same direction.
P.S. All disclaimers apply – this is not always the case and there are business folks who understand software and there are technical people who get the business and all that stuff. The description applies to many typical IT shops, not all. I’m also saying only that Agile makes it easier – not that it is the panacea to all software development problems.
 The Fifth Discipline by Peter M. Senge