s-expressions

Amit Rathore blogs about software development

Posts Tagged ‘process’

Design for throwability

Posted by Amit Rathore on January 16, 2009

Build one to throw away, you will anyhow. — Fred Brooks

Fred Brooks said what he did many, many years ago but it is probably just as true today. How many times have you and your team gotten a few months into a project only to realize all the design mistakes you made? Ask any engineer, and they’ll tell you they would build it right the second time.

This is just reality, the nature of discovering the complexity of the domain or the technology or the usage pattern or whatever else you didn’t know about when you started.

On the other hand, there’s this [what Joel Spolsky says about rewriting software] -

It is the single worst strategic mistake that any software company can make — Joel Spolsky

So… what gives? The answer, IMHO, is basically two things -

1. Understand and internalize the idea of the strangler application

2. Architect your system in such a way to support strangling it later

In essence, this means that because it would be a bad idea to rewrite the entire system from scratch, it must be built in a way so as to enable swapping out components of it as they are rewritten (or perhaps heavily refactored).

The architecture must draw from an approach called  concurrent set-based engineering (CSBE) – and indeed, sometimes each logical component would have more than one implementation. At Runa, two components of our system actually have two implementations each. And in each case, they’re both running in production – in parallel.

The way we accomplish this is through very loose-coupling. Additionally, because we take a very release-driven approach to our software process, our architecture evolves according to our current needs… and we refactor and extend things as new requirements are prioritized. At all times, despite our super-short release-cycles, our goal is to always have a version of the system in production. Whenever our pipeline tells us that a peice of the existing design may not work in the long term, we start to work on the replacements – more than one, and in parallel.

We run them in what I’ve been calling shadow-mode. This implies that its not quite part of the official system, but is running in order to prove some design hypothesis. Once everyone involved is satisfied with the results, we pick the most suitable sub-system and decommission  the other contenders (including the old one). At Runa, we achieve much of our inter-component loose-coupling via messaging (our current choice is RabbitMQ).

To summarize – we design everything  with one over-arching goal in mind  – the thing will be thrown away someday, and be replaced with another. As I said before, this enforces a few things -

1. Loose coupling

2. Clear interfaces between components

3. Good automated system testing!

About that last point – because we have many moving parts, functional testing becomes even more important. We currently use Selenium for true functional testing (Runa is a web-based service) – and a variety of other home-grown tools for custom systems testing. Not only do automated system tests tell us that the collaborating set of components are working right, but they also allow us to change things with impunity – knowing that we’ll know if things break.

This thinking is what I’ve been jokingly calling Design For Throwability – and it’s been working rather nicely. It’s essentially a design philosophy that embraces CSBE – and is especially useful for small startups where everything is changing quickly – almost by definition.

Posted in Uncategorized | Tagged: , , , | Leave a Comment »

Against all Oddities

Posted by Amit Rathore on May 5, 2008

I just remembered, over this past weekend, that a long time ago I had submitted an article for the ThoughtWorks Anthology. I had actually gotten accepted for the book – with a sort of caveat. Martin Fowler told me that I’d have to improve the essay – to make it actually helpful to people reading it. Great advice, of course!

Except that I’m way too lazy and couldn’t be bothered.

So here it is, in it’s original unedited glory. Against All Oddities by Amit Rathore. I’m also too lazy to convert it into HTML, let Google do that, I say!

Posted in Uncategorized | Tagged: , , | Leave a Comment »

Perfect vs. Shipped

Posted by Amit Rathore on March 31, 2007

I overheard an amusing comment in my team-room the other day, I think it might have been Kris Kemper who said it – “Anyone who knows Ruby On Rails has a half-done personal project that’s going nowhere”. How true. I have at-least three.

The thing is, in my mind I’m always envisioning these grand cathedrals, and even when I do start work on any one of them, I never seem to quite finish them. Or I don’t complete everything properly (or quite enough to be production-ready), and the application is never quite done.

I think it has to do with a lack of focus. I find myself thrashing between the hundreds of things that interest me, and I end up with a ton of unfinished work. I’m very much into lean software methods, and I know that all I’m doing by operating this way is creating a lot of inventory. I seem to be able to use lean and other workflow management techniques at work, but in the world of my personal projects, I seem to be at a loss.

Sometimes, it has to do with trying to get everything perfect. After all, since it is a personal project, I feel like I don’t have a delivery dead-line, so I can take the time to get it right. Which leads me down the rabbit-hole of perfection and cathedral building, with no real end. Cause there probably ain’t anything called perfection.

When consulting for our various clients, I’ve a clear idea in my mind about the compromises and trade-offs needed between design, architecture, refactoring, and delivery. And I aggressively do whatever might be needed to prod the folks along (be they developers, or product-owners) to get the thing done and into production. After all, there’s always another iteration coming up, and there’s always a next release.

So why the heck can’t I seem to do the same thing when I’m working on a nights-and-weekends project?

Posted in Uncategorized | Tagged: , , , , | 3 Comments »

The software development process and chaos theory

Posted by Amit Rathore on February 19, 2007

Over the weekend, I was talking to a non-software friend, and he asked what the big deal was about this whole software “engineering” thing. He’d been working in the manufacturing industry (he makes iron-ore furnaces) for many years, and said they’d solved the variability problem. They could make the furnaces within really small tolerances – and he could predict how long it would take. And, it would pass the levels of quality they’d set up for everything they made.

The fact of the matter was, as I explained to him, that it was impossible to control the effect of variability in the software process. Sure, Agile methods help many things – like setting up a constant heartbeat of 2-4 week iterations, providing a constant supply of jobs (stories) that need to be worked upon, ensuring constraints are identified and optimized etc.

However, even the smallest of changes can ripple into a totally different outcome. It’s like the whole thing about the butterfly beating its wings over the east cost of the United States – and it causing a hurricane in Japan. This is most evident if you try the thought experiment of having the same team build the same software twice in a row. The end result will be fairly different in each case – and certainly, the path taken will be very different.

Small variations, like some critical resource being on vacation on particular and different days, will cause changes in the schedule and dependencies that could potentially cause more significant changes in the overall project. Certainly, things like a different design (in the code) would cause different response times towards the various requirements as they came down the pipe. Different pairs of developers working on the same stories would result in different outcomes – maybe even different bugs. Heck, even the same developer coding the same thing twice would do it differently (right, guys?)

Given that these types of changes are arbitrary and therefore bound to happen, it is unreasonable to expect that software can be created (and the process controlled) to produce completely predictable results.

Just something to think about when trying to understand software development! And yes, I know it’s called complexity theory now, but chaos still sounds cooler.

Posted in Uncategorized | Tagged: , , | 4 Comments »

Ruby on hiring Java developers

Posted by Amit Rathore on March 1, 2006

I thought of one more use of Ruby. If you want to hire Java developers, hire ones who know Ruby. (Or Python (Definitely LISP))

Posted in Uncategorized | Tagged: , , , | Leave a Comment »

 
Follow

Get every new post delivered to your Inbox.

Join 1,586 other followers