Roles & Responsibilities

When I was a junior-level programmer, I felt like I had one responsibility - be good at programming.

Later in my career, once I became a lead developer, I learned that I now had another responsibility - deliver a project on time.

Still later, as a software architect I picked up another responsibility - to make it easier for the team of developers to be successful.

Today, I recognize another responsibility - deliver value to the business.

In the above chart, blue represents focus on Self, red is focus on the Team, and yellow is focus on the Business.

What does your chart look like?

How building a bridge is the same as building software

This is not the simplistic analogy you may be expecting.

Where I live in Minneapolis MN, we have a high profile bridge project going on right now, due to the tragic collapse of the previous structure. If you look at the project page on the state's website, you can see the following "features of the new bridge":

• 100-year life span
• 10 lanes of traffic, five in each direction—two lanes wider than the former bridge
• 189 feet wide—the previous bridge was 113 feet wide
• 13 foot wide right shoulders and 14 foot wide left shoulders, the previous bridge had no shoulders
• Light Rail Transport-ready which may help accommodate future transportation needs
• Design-build project complete in 437 days.
• Designed to be aesthetically pleasing and fit in with its environment

These are the most high-level (public) stakeholder values for the new bridge. From an engineer's perspective, they are the constraints under which the bridge must be delivered. In addition, we see an architect's rendition (picture) of what the new bridge will look like. This is also a constraint - an engineer cannot add things that will substantially change the appearance of the bridge.

Now, we can be sure that the design of the bridge was a collaborative effort of a team of people. Engineers, Marketing, Architects and the client. Is the design ongoing as they build the structure? Yes - they are using a growing technique known as Design-Build - they purposely start construction before the design is complete.

At first glance, design-build might sounds like a simple case of parallel work - one team is working on designing just-in-time, and another is working on the construction. In practice though, there is a collaborative environment that reportedly results in avoidance of disputes, faster project delivery, and less need for project management oversight.

The Analogy
So how is this similar to building software?

Firstly, the process of programming it is like the design of a bridge - it is the bringing together of people in different roles to creatively find ways to build the end result. Ideally, development involves a lot of Thinking, Talking, and Tweaking, just like a bridge design. In design, we often find that two heads are better than one. Pair-programming has been suggested as one way to do this in software development. Of course, we have many other collaborative techniques to communicate and discuss design.

Like a bridge design, the output of building software can be represented by piles of paper. The bridge has drawings, engineering specifications and requirements. A program has something better though - its code. (No, I'm not arguing that "the code is the design". I'm just saying that the code "is a representation of the design"). The code accurately describes the parts of the design that it touches.

This "programming code = bridge design" point is key to what I'm trying to convey - the process of programming produces a design output, not a product. The final product is the result of implementing that design (just as the bridge itself is the result of implementing its design).

Specifically, the "building" is the deployment of software in its final environment. Deployments are where the "tires meet the road" - they are the intersection of the design with reality (just like construction). Mostly, the design holds up and does not need tweaking after deployment. Sometimes though, the harsh lights of reality expose the hidden flaws in the design. (In light of that, it is best to expose an application to its first deployment as soon as possible).

Some software groups have QA (quality) departments. Historically, these departments have taken the role of performing trial deployments - they will take the software, and expose it to a simulation of the real environment. Large construction projects also have this role - an independent group audits the designs, with the hope of spotting problems that would cause a problem when the construction occurs.

Finally, we find that the best way of constructing a large bridge project is to simultaneously design and build. The analogy for software is small frequent releases. Research and experience has shown this to be a good way deliver quality software that meets the requirements.

Conclusion
If we accept that building a bridge and building software are similar (they contain the same basic steps), then we can use that information to produce some interesting insights:

• That thing we need to do before developing is "architecture" - There is a fine distinction between architecture and design. The way I like to define it is that architecture describes the parts are visible from the outside, and design describes the inside. A bridge architect is able to construct a working model and rendition of the outside of a bridge without the full engineering specs. To do this, he needs to take into account all of the stakeholder values. Similarly, we need to be able to draw the edges of a software application before we start - we need to understand how the software will interact with the outside world, and how the outside world will interact with the software.
• QA is a misnomer - the primary purpose of a separate QA department should not be to assure quality. We can get quality in better ways than that. The purpose of the QA department should be to validate the design of the software, by simulating real environments. Many QA professionals already know this, of course.
  

This blog post is inspired by a set of three essays by Jack W. Reeves.

Quote of the Day - John A. De Goes on Teams

Overheard on the XP list, John A De Goes describing his attitude towards teams and processes.

There's a growing realization that companies, organizations, and teams, they're all comprised of humans, and being a human is radically different than being a machine. You can't engineer something comprised of humans -- you have to grow it, nourish it, cultivate it. You can fix a broken machine by following a recipe, but you can't fix a group of people that way. Prescriptions for success amount to nothing, because they ignore the human component, which, when all is said and done, is the most significant factor determining success -- development methodology be damned.

Current thoughts on Estimation and Planning

Several leaders in their fields have been giving a great deal of thought lately to the state of planning and estimation, and how it relates to agile and lean projects.

Alistair Cockburn said:

The purpose of planning to be able to answer the question “With what we NOW know, what will the most useful system we can create look like on THAT date?”

Glen Alleman said:

The primary function of project management is to ensure that the project is implemented to meet the established budget, schedule, safety, and performance requirements to satisfy its objectives.

Corey Ladas said:

When we agree to take on a work request, we intend to deliver it within n days

There seems to be a general idea that the concepts of Lean can be applied very well to the process of software development, but it leaves unanswered the question of how to plan a long term project.

Agile and Lean both take the view that we will apply some fixed team size to the software, and we will deliver as fast as possible, and allow as much flexibility as possible to the client's ability to adjust the course (scope) of the project. This is undeniably the current most efficient way to produce software, but that does not mean that it is always the right choice.

The core of the problem is that detailed estimation is a wasteful effort - it does not add business value. So we can deliver more efficiently if we do not estimate. But we have to balance that against the requirements of the business, which has some objective that they are hoping to meet by some date. As the implementors of the solution, we are obligated to be able to tell the business the "when" of the delivery, and they are obligated to tell us the "what". Unfortunately, the exact "what" always changes. So even if we estimate the "when", we can very easily be wrong. Its a vicious circle.

In theory, Agile handles long term estimation better than Lean. This is because teams can estimate backlogs at a high level, months (but not years) in advance. Lean is very "just-in-time", so it can only answer the question at the last instant. The best "lean" answer so far is that we can offer smaller commitments to minimal marketable features. That is, basically what the quote (above) is from Corey.

Neither Lean nor Agile answers the problem of the longer term, more complex project with multiple groups and critical paths and a well-defined scope. The scope is the primary problem - Lean and Agile will deliver faster, but they are not willing to tell the client what he wants to hear - that the particular scope will be completed 2 years from today. They can still give some very attractive answers though - Alistair Cockburn has a list on his wiki.

My conclusion? There is no question in my mind that traditional planned projects are suboptimal, but for now, they are still appropriate for many situations. First though, I would try to sell the client on one of the approaches from Alistair's list - if they are willing to define what they want in a way that is compatible with Agile or Lean, then we can give them the result for much cheaper, or (more often) a better result for the same price.

Quote of the Day - TFS vs SVN

Heard on the altdotnet group. (SVN = Subversion, TFS = Team Foundation Server, both of which are primarily source control systems):

Oh, and as someone who is doing a lot with both SVN and TFS recently. I would trust SVN much more than TFS.

SVN make me think: Wow, someone special has thought about doing this.
TFS make me think: Wow, someone... special has thought about doing this.

- Ayende Rahien (aka Oren)