Thoughts in the Wilderness

This is not the simplistic analogy you may be expecting.

Where I live in Minneapolis MN, we have a high profile bridge project going on right now, due to the tragic collapse of the previous structure. If you look at the project page on the state’s website, you can see the following “features of the new bridge”:

• 100-year life span
• 10 lanes of traffic, five in each direction—two lanes wider than the former bridge
• 189 feet wide—the previous bridge was 113 feet wide
• 13 foot wide right shoulders and 14 foot wide left shoulders, the previous bridge had no shoulders
• Light Rail Transport-ready which may help accommodate future transportation needs
• Design-build project complete in 437 days.
• Designed to be aesthetically pleasing and fit in with its environment

These are the most high-level (public) stakeholder values for the new bridge. From an engineer’s perspective, they are the constraints under which the bridge must be delivered. In addition, we see an architect’s rendition (picture) of what the new bridge will look like. This is also a constraint – an engineer cannot add things that will substantially change the appearance of the bridge.

Now, we can be sure that the design of the bridge was a collaborative effort of a team of people. Engineers, Marketing, Architects and the client. Is the design ongoing as they build the structure? Yes – they are using a growing technique known as Design-Build – they purposely start construction before the design is complete.

At first glance, design-build might sounds like a simple case of parallel work – one team is working on designing just-in-time, and another is working on the construction. In practice though, there is a collaborative environment that reportedly results in avoidance of disputes, faster project delivery, and less need for project management oversight.

The Analogy
So how is this similar to building software?

Firstly, the process of programming it is like the design of a bridge – it is the bringing together of people in different roles to creatively find ways to build the end result. Ideally, development involves a lot of Thinking, Talking, and Tweaking, just like a bridge design. In design, we often find that two heads are better than one. Pair-programming has been suggested as one way to do this in software development. Of course, we have many other collaborative techniques to communicate and discuss design.

Like a bridge design, the output of building software can be represented by piles of paper. The bridge has drawings, engineering specifications and requirements. A program has something better though – its code. (No, I’m not arguing that “the code is the design”. I’m just saying that the code “is a representation of the design”). The code accurately describes the parts of the design that it touches.

This “programming code = bridge design” point is key to what I’m trying to convey – the process of programming produces a design output, not a product. The final product is the result of implementing that design (just as the bridge itself is the result of implementing its design).

Specifically, the “building” is the deployment of software in its final environment. Deployments are where the “tires meet the road” – they are the intersection of the design with reality (just like construction). Mostly, the design holds up and does not need tweaking after deployment. Sometimes though, the harsh lights of reality expose the hidden flaws in the design. (In light of that, it is best to expose an application to its first deployment as soon as possible).

Some software groups have QA (quality) departments. Historically, these departments have taken the role of performing trial deployments – they will take the software, and expose it to a simulation of the real environment. Large construction projects also have this role – an independent group audits the designs, with the hope of spotting problems that would cause a problem when the construction occurs.

Finally, we find that the best way of constructing a large bridge project is to simultaneously design and build. The analogy for software is small frequent releases. Research and experience has shown this to be a good way deliver quality software that meets the requirements.

Conclusion
If we accept that building a bridge and building software are similar (they contain the same basic steps), then we can use that information to produce some interesting insights:

• That thing we need to do before developing is “architecture” - There is a fine distinction between architecture and design. The way I like to define it is that architecture describes the parts are visible from the outside, and design describes the inside. A bridge architect is able to construct a working model and rendition of the outside of a bridge without the full engineering specs. To do this, he needs to take into account all of the stakeholder values. Similarly, we need to be able to draw the edges of a software application before we start – we need to understand how the software will interact with the outside world, and how the outside world will interact with the software.
• QA is a misnomer – the primary purpose of a separate QA department should not be to assure quality. We can get quality in better ways than that. The purpose of the QA department should be to validate the design of the software, by simulating real environments. Many QA professionals already know this, of course.

This blog post is inspired by a set of three essays by Jack W. Reeves.

1. In some older languages, the runtime was not able to expose a call stack when exceptions occurred. This led to programmers spattering the code with exception handling-code, so that they would have a better idea of where exceptions occurred.
2. Historically, programmers have sometimes used exceptions to communicate status.

The above 2 ways of doing things are no longer appropriate.

First, in a language which exposes an exception call stack (like C# or VB.NET), the only place you need an exception handler is at the top-most point of a thread. So, you need a try…catch around your application entry point (Main sub), and around every new thread you manually spawn.

Second, using exceptions to communicate status is a terrible practice. I cannot articulate why really. Perhaps because it forces you to code everything very defensively, which obscures the intention of the code. In any case, experience shows that things are much simpler if you follow the rule that exceptions are only for exceptional scenarios. In other words, do everything you can to avoid having to trap exceptions.

This is harder than it sounds. It pairs well with the “fail fast” rule though. You can implement this rule by validating parameters to your methods. If they do not meet your expectations (e.g. something is NULL when it should not be), then throw an exception. The intention of the exception is always to indicate an invalid system state. (An invalid system state is unpredictable and dangerous, and should never occur – it is an exception).

It is disturbing for me when I review new code, and I notice that it is almost identical to similar code elsewhere. Usually, copy-and-paste programming is the cause.

Sometimes it is ok, because what is being copied is essentially configuration metadata. But mostly it is a bad thing.

I want to talk about a scenario that I see often, and I think is very common in software development.

We work on some problem domain where we will need multiple implementations. We gain enough understanding to output a version 1.0 implementation, wherein we develop some re-usable parts and some not-so reusable parts.

This is ok. We have learned, and when we come to doing a second implementation we will apply what we learned to make it even better. Or that is how it should be! But it does not happen.

What happens instead is that programmers love re-use (and why wouldn’t we – it makes our jobs seem easier). We love it so much, that we will use copy-paste to achieve it. That is, we will copy implementation 1.0 and then try to shove implementation 2.0 into that box.

Never mind that we do not understand the problem sufficiently to know if implementation 2.0 is sufficiently like implementation 1.0 to use the same box. We will copy the box, and then try and mold it to our needs.

This is a recipe for untidy, silly code that cannot handle the little edge cases that come up, because implementation 2.0 is never the same as 1.0.

So what is the solution?

Young grasshopper…forget re-use. It is a red herring. A diversion, an evil distraction. It is not achievable in the way you think.

Forget re-using implementation 1.0. Implementation 2.0 is a chance to start over with a clean slate. An empty page, a new design. The only re-use is in your head – refining and learning. Implementation 2.0 is your chance to apply what you have learned to the problem.

The secret you have to accept is this:

Increasing your understanding of the problem domain is the only way you will achieve sustainable re-use.

When someone (or groups of someones) increase their knowledge of the problem domain to the point at which they achieve a form of enlightenment, then sustainable re-use is not only possible – it is inevitable.

It may take 3 implementations before you achieve enlightenment, or it may take more. The simple lesson is this:

Don’t try to re-use existing code. What you want to re-use is an API – a way of thinking of the problem domain. Keep trying new things to improve the way you solve the problem. Re-use will find you when you are ready.

Debugging. The process by which a programmer discovers how his program works.

I am a hyperproductive programmer. That means I can output 3-20 times the work that an average programmer can. (Arrogant? Maybe. Still true though).

One of the biggest reasons I am productive is that I make a habit of not “debugging”. This is my theory:

Debugging code is always wasted time

The only output of debugging is a greater understanding of the code. But there are better ways to understand code. I can read it. I can refactor it so as to make it easier to read. Leading to Corollary one:

Code reading and refactoring are more important skills than debugging.

Time spent debugging is not only wasted, it smells of poor code quality. If I lack understanding, that means that the code was too hard to read.

Programmers that do Test-Driven development understand this productivity boost. Once you write unit tests, you find that you no longer have to debug. Leading to Corollary two:

Improved up-front quality leads to less debugging.

Moving on. Maintainability. An ugly word. A nicer word is Soluble, (or grokkable). My productivity is far higher when I am dealing with code that I “grok”. Put me on a new project, and it will take me a while to come up to full speed. Much of that time will be debugging, refactoring, or reading code. Leading to Corollary three:

Solubility of code has a direct impact on time spent debugging.

(That is so obvious that it may be a truism).

In summary – debugging is a symptom caused by underlying causes of poor code quality, poor programmer skills, and code that is hard to read.

If you notice yourself, or other programmers debugging code, then ask yourself – which combination of the above is a problem? Then fix it.