Custom Software Scope Changes (Not)

[If you have never been involved in a custom software engagement (e.g., on the software builder side, customer buyer side, contract law side, etc.), there is no need to read this post.]

I spent roughly 22 years of my life building somewhere between 150-200 custom software systems ranging from accounting systems, llama genealogy systems, labor reporting systems, airport profitability systems, huge consolidation consumer data warehouses, and sewer flow modeling systems to the first versions of NORA.

There was one particularly interesting lesson from these days:

“Custom software change orders CLAIMED by builders … are actually more often changes in understanding that come to light as the visions between parties are forced to reconcile.”

Here is how this goes: The buyer describes what they want to the builder.  The builder captures this vision in the form of requirements, features, design layouts, etc.  This exchange of vision continues until the builders and buyers think they are in synch.  The builder starts to build.  The buyer starts to see the system come to life.  The buyer notes that what he sees is not quite what he had in mind and calls for some adjustments.  The builder calls this a change order.

What is really happening is the visions between parties are being forced to reconcile as the system takes shape in the real world.

In my book, this is not a scope change.  This only demonstrates that both parties had something different in mind on day one.

There is only one way to safely avoid this all too frequent problem and that is blueprints.  And I am not talking about a few screen sketches and report headers.  I am talking about robust architectural plans: complete screen layouts with error messages, full report mock-ups containing realistic data and sub-totals.  I am talking about a degree of design completeness that the specifications could be used to simulate (e.g., in paper form) the use of the whole system by each type of user.

No one builds a high-rise building let alone a doghouse without blueprints.  Otherwise, there is an extraordinary risk of a) an abandoned project or b) the creation of a Dr. Seuss “Who House.”

On a related note: some think that rapid prototyping of big complex systems can shortcut a comprehensive design phase.  With few exceptions, I think this is ridiculous.  [I will elaborate on this is a coming post.  But in the mean time, as a teaser, I have a single test question that determines if a custom software system has been well-conceived.  Once the system is all done, knowing what you know now, would you have built it the same way?]

Is the custom software world still this way?

RELATED POSTS:
Super Consultants

Scalability and Sustainability in Large Information Sharing Systems

Comments

[matt mcknight]

I think the underlying issue you aren't mentioning here is that software estimation is a broken process. If you look at Steve McConnell's recent work in this field, a lot of contracts are written very early in the cone of uncertainty, when neither customer nor builder has a great idea of what is wanted. His solution to this is to use two phase contracting- an phase one contract that is used to further define and scope the existing work. That sort of fits with what are writing, but is too risky for me.

Rather than demand "blueprints", why not demand a system that allows for rapid construction and reconstruction? The construction analogy falls apart for software, because it imposes the false constraint of the building. I can sit with users and analysts and build a working web application, minus the more complex logic, more quickly than I could in the paper form you suggest. I have seen reams and reams of paper generated on projects that never deployed, while things that start smart and start small are often able to be nurtured into successful systems.

While you say that risk of an abandoned project or a Who House is higher if you don't do blueprints- this is not true in practice. Very few software projects are rejected at the architectural design phase. A better way to reduce this risk is to begin to deliver value earlier in the project. If you start with something that works you get valuable feedback from people that are actually using the system, which vastly exceeds the value of people's imagination of how they would want to do something. It also starts delivering ROI sooner.

I 100% agree with you that getting the user interface/user experience right is key, I just think that modern web tools can make it happen in a different way. Have you read "Getting Real" from 37 signals? http://gettingreal.37signals.com/toc.php

One secret of all of this is building on top of simple, proven application architecture and framework for rapid development.

Here's a problem for you to consider- looking at the evaluation of off the shelf software that is often done via market surveys, how often is the programmability of the software considered? I see the wrong choices being made again and again because no one considers that the humane interface of the application in regards to the developers that have to hook into the rest of the system.

[Ian Story]

I think there are two problems with the blueprints approach (or the traditional approach of specifications) -
The first problem being cost and time to put them together (which of course, others will argue, and I will agree, that it pays off in the long run by reducing work and so forth...but if the upfront cost of specs is going to be half the project cost, that can be a tough pill to swallow for many customers.
The second problem, more with the traditional approach of specs, is that the specs are often written by technology types, and the business doesn't understand them (or in many cases, doesn't read them) and that causes gaps.
Perhaps the blueprint idea helps there, as long as you figure out a way to do it very, VERY quickly so as not to create excessive upfront costs.
As always, great post Jeff (and props to Matt for mentioning the 37 Signals manifesto :)...am enjoying the heck out of the blog after meeting you a few weeks back in Seattle. Good times. --Ian

[Larry Moody]

Hallelujah Jeff!

I could not agree more with your post; Customer Software Scope Changes (Not).

Coincidentally, I read this survey from CodeProject.com just prior to reading your article…

http://www.codeproject.com/script/Surveys/Results.aspx?srvid=781

I don’t understand how this fly-by-the-seat-of-your-pants SDLC became the norm, but it happened.

I was beginning to think that I missed the latest new methodology… You know, the “WRRRTO” method: (Write, rewrite, rewrite, rewrite, throw out).

[James Taylor]

Posted a response and the track back does not seem to have worked so here's the link:
http://smartenoughsystems.com/wp/2008/04/24/scope-business-rules/

JT

[Tom Grimshaw]

G'day Jeff,

I agree with the principle you espouse, that more detail in the specification will flush out the lack of duplication between the two parties, but my opinion is that blueprints are not the only answer.

There are a number of methods of fleshing out the details. Your experience has obviously been with people and systems for which blueprints lend themselves as a workable solution.

The reason Agile software development practices arose was that 50% of the software created with the Waterfall method was never used.

Some of our clients can only fully understand that their concepts have been completely understood when they have a piece of software they can play with, if only at a rudimentary level. I echo Matt's comments about being able to create an interface in less time than comprehensive documentation. I add that the certainty we are on the same page would be far higher with the GUI than the docs.

[Bob P]

I'm not sure what type of software development projects the other commenters have been involved with, but unless your user community is very small, you cannot simply 'sit down' with a bunch of users and expect everything to be perfect, you can't just skip the process of creating documentation, and you absolutely cannot bypass a critically important step called testing. That testing should involve usage scenarios, which mitigates the risk that two or more parties are not truly in synch (regardless of how early in the process the user/customer is engaged). Otherwise you get into the 'working as designed' vs. 'not working as intended' debacle. Those scenarios are part of the blueprints. And with the advent of virtual environments ('CAD for the masses'), even building blueprints don't need to be on paper.