Rapid Application Development: Fast Car No Map and 50 Laps to a “Who House”

[If you have never been involved in a custom software engagement (i.e., on the software builder side, customer buyer side, contract lawyers, etc.), there is no need to read this post.]

Rapid application development and prototyping have their place. One of these places is not the creation of complex, mission critical, ultra-high performance systems (i.e., space shuttles not mash-ups) – where it is essential that in hindsight the system appears well thought out, explainable, maintainable, etc. The simplest post-project assessment being, “Knowing what I know now, I would have built it exactly this way!”

The only method I am aware of that has a reasonable chance of making the above claim involves building a system based on stable, pre-defined, well-conceived blueprints. When operating off tight specifications, what you end up creating is virtually no different that what you were envisioning at the start … including the very first brick laid.

If you start building something without a precise road map, every time you have a design revelation, resource conservation drives one to make compromises. It is in this way that the air conditioning unit ends up on the roof of the car, the stereo mounted to the back of the driver’s seat, and so on. What rapid application development environments can lead to is the fast, iterative (50 laps) construction of a  Dr. Suess "Who House" or a Rube Goldberg-like contraption.

My primary issue with rapid prototyping, rapid application development, iterative development, and the like is the creation of valuable artifacts that one is hesitant to later discard at a whim. On this path, it becomes more prudent to perform a least-effort modification of A to become B. This compromise is compelled by the urge to preserve earlier investment … even if the best tact involves utterly discarding A and constructing B from the ground up.

It is therefore my belief that the most valuable prototyping tools, rapid application development environments, etc. are those that leave the designer with a sense of “as close to zero investment.” Radical changes to the design, complete rethinking of a whole component, big shifts in database schema, wholesale rework to the GUI, etc., need to be absorbed in the design phase without an iota of grief.

[One of these days maybe I’ll blog about Computer Aided Software Engineering (CASE) technology. During one era of my life I spent eight years building such technology to automate the blueprint process; hence this subject remains very dear to my heart.]

RELATED POSTS:

Super Consultants

Custom Software Scope Changes (Not)

Scalability and Sustainability in Large Information Sharing Systems

Comments

[James Taylor]

One of the best comments on this topic was made by a colleague at Ernst and Young when he and I were both working on methodology and CASE tools. He compared software to buying a house - prototyping is like visiting houses, specification/blueprints like writing a list of requirements for your realtor. Just visiting houses until you find one you like is not going to work but any attempt to right down exactly what you want without visiting any houses is not going to work either. So, specify enough to visit the right kinds of houses and expect those visits to keep changes your requirements until everything stabilizes.
You must, in software, build specifications and blueprints but you must also prototype/simulate to make sure those blueprints are correct. CASE tools failed, in part, because they could not do the simulation piece.
JT
Author of Smart (Enough) Systems

[Jay Levitt]

[Disclaimer: I'm a big proponent of the agile philosophy, but I have yet to try it on a big, critical project. That said...]

Are you sure?

Every time I've done a large, complicated project using the old Big Requirements Up Front (BRUF) method, two things happened:

1. The planning process took so long that, by the time it was done, our requirements were out of date. This is what I call a "physics problem" - it can't be solved. Every day you spend working on the paper version/prototype is another 24 hours for the real world to change.

2. No matter how much experience we had with similar projects, and no matter how many experts and eyeballs we had working on the requirements, we always, always, always missed something major - something that became evident within a week or two of actually rolling out the system. People just don't use a prototype the way they'd use the real thing. They don't even use a beta test the way they'd do the real thing. You don't really know what's missing until you roll it out to the Big Lab.

Obviously, a lot depends on what tools you are using. If you're working in assembly, C, C++, even (especially!) Java, then yes - you have a huge investment in your current architecture, and you're going to try to get from A to B with the least possible pain.

The current trends in dynamic languages are towards frameworks and development tools that make it painless to get to B the *right* way - no compromises. You're spot on with the "as close to zero investment" comment; what you may not realize is that it's now possible to write the *actual application* with near-zero investment in its infrastructure.

If you can change anything you want at any time, you don't need fancy prototyping tools; the prototype is the software. You get it running, and then you get it right. This won't work for the space shuttle, or any other embedded, non-updateable apps. But it works fine for server software, and for client software that can download its own updates.

Considering how attuned you are to the psychology of software, I'm surprised to hear you trumpeting BRUF. Yes, perhaps if we had perfect tools and perfect testers, it would work better in theory. But we don't; we have human developers and human users. And they work better with the real thing in their hands.

[Girish]

Just came across your blog post, I had done a similar post a few days ago, BTW it was about the need for a easy to use RAD protoyping tool that could stand the test of time

http://fullpaisavasool.blogspot.com/2008/05/rapid-app-development-friend-or-foe.html

[Sean B.]

Having seen both approaches (BRUF & Agile) flounder and fail, I've come to the conclusion that a slavish devotion to any methodology leads to unavoidable compromises in planning, too much focus on the process rather than the project, and poor behaviors related to decision making and the inevitable project changes.

I too look forward to greeting our future automated overlords of software development, but seeing so little progress in the industry is discouraging. Are Intellisense and syntax highlighting the best we can do?