How building a bridge is the same as building software

This is not the simplistic analogy you may be expecting.

Where I live in Minneapolis MN, we have a high profile bridge project going on right now, due to the tragic collapse of the previous structure. If you look at the project page on the state's website, you can see the following "features of the new bridge":

• 100-year life span
• 10 lanes of traffic, five in each direction—two lanes wider than the former bridge
• 189 feet wide—the previous bridge was 113 feet wide
• 13 foot wide right shoulders and 14 foot wide left shoulders, the previous bridge had no shoulders
• Light Rail Transport-ready which may help accommodate future transportation needs
• Design-build project complete in 437 days.
• Designed to be aesthetically pleasing and fit in with its environment

These are the most high-level (public) stakeholder values for the new bridge. From an engineer's perspective, they are the constraints under which the bridge must be delivered. In addition, we see an architect's rendition (picture) of what the new bridge will look like. This is also a constraint - an engineer cannot add things that will substantially change the appearance of the bridge.

Now, we can be sure that the design of the bridge was a collaborative effort of a team of people. Engineers, Marketing, Architects and the client. Is the design ongoing as they build the structure? Yes - they are using a growing technique known as Design-Build - they purposely start construction before the design is complete.

At first glance, design-build might sounds like a simple case of parallel work - one team is working on designing just-in-time, and another is working on the construction. In practice though, there is a collaborative environment that reportedly results in avoidance of disputes, faster project delivery, and less need for project management oversight.

The Analogy
So how is this similar to building software?

Firstly, the process of programming it is like the design of a bridge - it is the bringing together of people in different roles to creatively find ways to build the end result. Ideally, development involves a lot of Thinking, Talking, and Tweaking, just like a bridge design. In design, we often find that two heads are better than one. Pair-programming has been suggested as one way to do this in software development. Of course, we have many other collaborative techniques to communicate and discuss design.

Like a bridge design, the output of building software can be represented by piles of paper. The bridge has drawings, engineering specifications and requirements. A program has something better though - its code. (No, I'm not arguing that "the code is the design". I'm just saying that the code "is a representation of the design"). The code accurately describes the parts of the design that it touches.

This "programming code = bridge design" point is key to what I'm trying to convey - the process of programming produces a design output, not a product. The final product is the result of implementing that design (just as the bridge itself is the result of implementing its design).

Specifically, the "building" is the deployment of software in its final environment. Deployments are where the "tires meet the road" - they are the intersection of the design with reality (just like construction). Mostly, the design holds up and does not need tweaking after deployment. Sometimes though, the harsh lights of reality expose the hidden flaws in the design. (In light of that, it is best to expose an application to its first deployment as soon as possible).

Some software groups have QA (quality) departments. Historically, these departments have taken the role of performing trial deployments - they will take the software, and expose it to a simulation of the real environment. Large construction projects also have this role - an independent group audits the designs, with the hope of spotting problems that would cause a problem when the construction occurs.

Finally, we find that the best way of constructing a large bridge project is to simultaneously design and build. The analogy for software is small frequent releases. Research and experience has shown this to be a good way deliver quality software that meets the requirements.

Conclusion
If we accept that building a bridge and building software are similar (they contain the same basic steps), then we can use that information to produce some interesting insights:

• That thing we need to do before developing is "architecture" - There is a fine distinction between architecture and design. The way I like to define it is that architecture describes the parts are visible from the outside, and design describes the inside. A bridge architect is able to construct a working model and rendition of the outside of a bridge without the full engineering specs. To do this, he needs to take into account all of the stakeholder values. Similarly, we need to be able to draw the edges of a software application before we start - we need to understand how the software will interact with the outside world, and how the outside world will interact with the software.
• QA is a misnomer - the primary purpose of a separate QA department should not be to assure quality. We can get quality in better ways than that. The purpose of the QA department should be to validate the design of the software, by simulating real environments. Many QA professionals already know this, of course.
  

This blog post is inspired by a set of three essays by Jack W. Reeves.

The 8 T's in a developer's work-day

All developers do the following things in various proportions through the day (starting with T, for fun):

• Typing
• Thinking (design)
• Twiddling thumbs (waiting)
• Tinkering (experimenting)
• Trying (debugging)
• Talking (design, analysis)
• Testing
• Tweaking (refactoring)
  

Some add no direct value (Trying, Typing, Twiddling thumbs, maybe Tweaking) and some add value in moderation (the rest).

The value in this list is that you can use it to motivate the validity of various practices. For example, doing TDD will increase Typing, but it will almost eliminate Trying, significantly improve the efficiency of Thinking and do some Testing for you. For many scenarios, this means that the benefits outweigh the costs.

Filling in timesheets will increase Typing, but will not improve anything else.

Pair programming will increase Talking and Thinking, but maybe decrease Typing (only one person can type at a time).

Agile needs Architecture

Consider TDD (test-driven-development). TDD is a great design technique. It creates systems that are wonderfully decoupled. It lets you build something very quickly and effectively. It allows developers to transcend their own limitations, and results in a system that is more than the sum of its parts. Beautiful.

TDD software is an evolved work of art, beautiful like an organically grown crystal.

Compare that with a system that is designed by a software architect. Architecture is about drawing lines, and encapsulation. It is about understanding current and future needs and using that understanding to define the edges of a system, how they should interact, and which pieces should be interchangeable.

Software designed by an architect is like a piece of machinery. It has lots of moving parts, which interact in well-defined ways.

A TDD system that is under the guidance of an architect will be better than one which is not. Similarly, an architected system that is implemented using TDD will be better than one which is not. TDD and architecture are complementary techniques.

At a small scale, the distinction is not as important. For a small system, TDD can evolve something very nice, with limited architectural input. At larger scales however, someone needs to be looking at the big picture. You simply cannot evolve the design of a machine, or a house.

Getting to my point...

It bothers me that none of the Agile techniques stress the need for the role of architect. They assume that you can put together a group of equally skilled programmers and the design will evolve. This is true to an extent, but TDD and similar techniques can only take you so far. At some point, you need someone who can see the "big picture".

Productivity secrets - Debugging

Debugging. The process by which a programmer discovers how his program works.

I am a hyperproductive programmer. That means I can output 3-20 times the work that an average programmer can. (Arrogant? Maybe. Still true though).

One of the biggest reasons I am productive is that I make a habit of not "debugging". This is my theory:

Debugging code is always wasted time

The only output of debugging is a greater understanding of the code. But there are better ways to understand code. I can read it. I can refactor it so as to make it easier to read. Leading to Corollary one:

Code reading and refactoring are more important skills than debugging.

Time spent debugging is not only wasted, it smells of poor code quality. If I lack understanding, that means that the code was too hard to read.

Programmers that do Test-Driven development understand this productivity boost. Once you write unit tests, you find that you no longer have to debug. Leading to Corollary two:

Improved up-front quality leads to less debugging.

Moving on. Maintainability. An ugly word. A nicer word is Soluble, (or grokkable). My productivity is far higher when I am dealing with code that I "grok". Put me on a new project, and it will take me a while to come up to full speed. Much of that time will be debugging, refactoring, or reading code. Leading to Corollary three:

Solubility of code has a direct impact on time spent debugging.

(That is so obvious that it may be a truism).

In summary - debugging is a symptom caused by underlying causes of poor code quality, poor programmer skills, and code that is hard to read.

If you notice yourself, or other programmers debugging code, then ask yourself - which combination of the above is a problem? Then fix it.

Software Architect Definition

Today I came up with a short definition of a software architect that I think is broadly applicable (to myself and others that perform the role).

"The role of a Software Architect is to recognize the edges of systems, communicate them, and define their APIs."

"recognize the edges" - By edges, I mean both vertical (more conventionally called layers) and horizontally (separate applications). The edges are not always easy to recognize. Lots of time there is some overlap in system functionality, and the role of the architect is to recognize those overlaps and envision the possibilities of resolving them. Sometimes this means inventing new systems to fulfill more specialized roles, and then retrofit existing systems to use the new specialized systems. Sometimes it means nothing other than putting procedures in place to limit the impact of the duplication.

"communicate them" - The edges are where the politics and the business of architecture reside. A good architect can help an organization discover more efficient ways of doing business, but they will have to communicate to make it happen. And that is at it should be - the architect is the technical expert, but the business people know the business, and change is only worthwhile if it has business value.

"Define their APIs". Good APIs are always designed. They never evolve by themselves. I know this because I have created some of my own, and I have listened to presentations from the people that designed the APIs for Java and .NET (probably the 2 biggest APIs around). Characteristics of a good API are:

• Easy to learn and use, even without documentation (discoverable)
  
• Hard to misuse
• Sufficiently powerful to satisfy requirements (but not much more)
  
• Easy to extend
• Appropriate to audience

The APIs within a system are low-level design. They are important to the maintainability of the system, but they can evolve by themselves, using techniques such as Test-Driven Design (TDD). Their audience is small and specialized. The architect has little business dictating this level of design, although they should certainly dictate which external APIs the developers will be making use of.

The APIs on the edges of a system are at a higher level. TDD (and use of design patterns) makes for bad "system" APIs, because they are not discoverable or appropriate to their audience. To use such an API, you typically need to create a few classes (maybe Strategies), or even implement an interface, and only then can you call a method which will do what you want. This is where the architect needs to step in and (writing code if necessary) ensure that the system can expose itself to the outside world in a proper way. There is clear business value in API design at this level.