Thoughts in the Wilderness

First, I should touch on the intent of good architecture. The intent is to build something of quality that will last a long time. The “something” we build will not be static – it will be changed, and should be amenable to those changes without loss of quality. Small applications are easy to replace, rather than change – so good architecture is most relevant to medium to large applications.

To review, my first law of good architecture is:

Identify all core services to the application. Code against the interface of the service you wish you had, not to the implementation of the one you actually have.

Now when I talk of services here, I am specifically *not* talking about SOA. I am talking about all the pieces of your application that are not business logic. In this context, a “core service” is pretty much everything that is not the business logic itself. This includes the entire user interface, the database, the file system, and the application settings.

User Interface
Lets talk about the user interface as a service. This is a little-known technique, so I’ll take the time to motivate it as best I can.

Consider this statement: Logic naturally wants to be at the points of control. By default, the main point of control of an application is its user interface. This is why it is so hard for developers to keep it out of there! For non-visual applications, this is still true – the logic wants to be on the edges. From an architectural perspective, this tendency is very dangerous – the user interface is the most likely part of the application to be discarded, and the hardest (practically impossible) to re-use.

One well-known technique for limiting the damage is layering of the user-interface on top of the application logic. With discipline, this can work well. However, good architecture does not assume discipline. It assumes team members of average talent at best, and structures the application so that they are as effective as possible. Layering is not the best answer.

Given that logic wants to be at the point of control, we can make a conscious decision to put the application logic in control. This will make the other parts to the application subservient to the application logic. As it turns out, that is the definition of a service – a part of the application that is subservient to another.

Another important characteristic of a service is that it has a well-defined API. So well defined in fact, that we can define its interface, and code against that interface rather than the actual service. So score 1 for the user interface as a service – it makes automated testing of business logic easy.

Of course, the user will still interact with the application – clicking on menus, entering data, and generally driving the flow of the application. However, they will be doing that within the context that the application logic has defined and supplied to the user interface.

In practice, this is a lot easier than it sounds. You can evolve the interface as you develop the application. Modern inversion of control techniques make it easy to inject the actual user interface at the time of execution, and to supply an appropriate context that the user interface can operate within.

The File System
Some pre-built services, such as the file system are very broad. Do we create an interface over that entire surface? No. We code against the interface we wish we had. The file system may provide the implementation, but we would be introducing unnecessary complexity if we dealt with the file system directly.

Settings
My own view is that you can combine settings with the user interface service. This is because settings can often be user-choices in one implementation, and settings in another, and hard-coded in yet another. There is no perceivable downside to having the user interface implementation control the settings.

The Database
It turns out, the most difficult aspect of the application to make into a service is the database. A database is like a pool of data. Most times, the interface we wish for is to be able to scoop up the data with a bucket, play with it, then throw the data back into the pool. A simple data access layer can be good enough for this.

Sometimes we want more – for example, we may want to have data access run on a different application server, or be scaled across multiple servers. We may want to provide the ability to have the application run disconnected from a server. Or we may want to totally insulate the application from the data structure or vendor. These are all up-front choices we must make. All come with a cost. In the more expensive cases, the interface we wish for will be more service-like than a simple data access layer would.

If we do have to make data access into a service, we should still be sure to make it the service we wish we had. This implies that design of the service should be driven based on the needs of the application.

Conclusion
Good architecture puts the important logic at the center and treats the less important logic as subservient (services). We code against the services we wish we had, because to do otherwise introduces artificial complexity. (The implementation of the services can take care of translating back to the reality of the underlying provider).

“User Interface as-a-service” is a new concept. I have implemented it with success, although I didn’t understand it then as well as I do now. Others have too – Cockburn’s hexagonal architecture is a similar concept to what I have described. I think I will be writing more about it in later posts, because I have treated it too high-level here. People will want to know how to actually do it before they believe it is a good idea.

It is disturbing for me when I review new code, and I notice that it is almost identical to similar code elsewhere. Usually, copy-and-paste programming is the cause.

Sometimes it is ok, because what is being copied is essentially configuration metadata. But mostly it is a bad thing.

I want to talk about a scenario that I see often, and I think is very common in software development.

We work on some problem domain where we will need multiple implementations. We gain enough understanding to output a version 1.0 implementation, wherein we develop some re-usable parts and some not-so reusable parts.

This is ok. We have learned, and when we come to doing a second implementation we will apply what we learned to make it even better. Or that is how it should be! But it does not happen.

What happens instead is that programmers love re-use (and why wouldn’t we – it makes our jobs seem easier). We love it so much, that we will use copy-paste to achieve it. That is, we will copy implementation 1.0 and then try to shove implementation 2.0 into that box.

Never mind that we do not understand the problem sufficiently to know if implementation 2.0 is sufficiently like implementation 1.0 to use the same box. We will copy the box, and then try and mold it to our needs.

This is a recipe for untidy, silly code that cannot handle the little edge cases that come up, because implementation 2.0 is never the same as 1.0.

So what is the solution?

Young grasshopper…forget re-use. It is a red herring. A diversion, an evil distraction. It is not achievable in the way you think.

Forget re-using implementation 1.0. Implementation 2.0 is a chance to start over with a clean slate. An empty page, a new design. The only re-use is in your head – refining and learning. Implementation 2.0 is your chance to apply what you have learned to the problem.

The secret you have to accept is this:

Increasing your understanding of the problem domain is the only way you will achieve sustainable re-use.

When someone (or groups of someones) increase their knowledge of the problem domain to the point at which they achieve a form of enlightenment, then sustainable re-use is not only possible – it is inevitable.

It may take 3 implementations before you achieve enlightenment, or it may take more. The simple lesson is this:

Don’t try to re-use existing code. What you want to re-use is an API – a way of thinking of the problem domain. Keep trying new things to improve the way you solve the problem. Re-use will find you when you are ready.

Traditional OO lore teaches us that objects are things that have both data and behavior. Blindly following this rule can lead us to make poor design choices, especially around what many refer to as “business objects”.

The pattern is that these objects already have data, so we seek to add behavior as well. In this way we can feel happy and content that we have a true “object”, and we are successful OO programmers.

The problem is that adding behavior as a sort of “suffix” to an object is ignoring a more important aspect of objects, which is that they should do one thing, and do it well. Add too many “suffix” behaviors, and pretty soon you can have a tightly coupled bowl of spaghetti.

This is not just theoretical – I have seen it happen, more than once. I’ve even been guilty of it.

So what is the solution? When we have classes that are primarily data, should we resist adding behavior?

My answer is “it depends”. To understand why, we need to take a small detour into API design…

Sometimes, programmers expect things to be a certain, simple way. They do not want to ask a FactoryLocator for an IObjectPersistorFactory, use that to get an IObjectPersistor, and finally tell the IObjectPersistor to Save their object to the database. They just want to write:

myObject.Save()
or
myObject.Load(id)

This ActiveRecord implementation is easy to write and easy to read. In short, it is good because it is a nice API for the client of the object. It has drawbacks (no transaction support, high risk of coupling to database). But in many systems, this API will be sufficient.

So the ActiveRecord “suffix” is mostly ok. What other behaviors can we add? How about validation? The save method should probably validate before it saves, so as to ensure we have good data in the database. How about some initial field values for new objects? And some event driven behavior – let field A be defaulted when field B changes? And we need properties for other objects. MyCustomer.Address.ZipCode works real nice. We can even lazy-load the Address property. Not too hard.

Hmm. Question. If we save the Customer object, should the Address save too? Probably. So we need to add some more code to the Save method for that.

etc. etc.

You get the picture (I hope). You can create a perfectly functional system in this way, but the coupling of all functionality to a single class will make it difficult to change in any substantial way. It will also have poor quality, because we are ignoring several key principles, such as DRY and Open-Closed.

There is only one way in which you can mitigate the problem. Use code-generation to generate your “business object” implementations. This mitigates quality problems substantially (DRY does not apply to generated code). It also forces you to either state some things declaratively (such as required fields), or else move them into their own dedicated area.

Today I came up with a short definition of a software architect that I think is broadly applicable (to myself and others that perform the role).

“The role of a Software Architect is to recognize the edges of systems, communicate them, and define their APIs.”

“recognize the edges” – By edges, I mean both vertical (more conventionally called layers) and horizontally (separate applications). The edges are not always easy to recognize. Lots of time there is some overlap in system functionality, and the role of the architect is to recognize those overlaps and envision the possibilities of resolving them. Sometimes this means inventing new systems to fulfill more specialized roles, and then retrofit existing systems to use the new specialized systems. Sometimes it means nothing other than putting procedures in place to limit the impact of the duplication.

“communicate them” – The edges are where the politics and the business of architecture reside. A good architect can help an organization discover more efficient ways of doing business, but they will have to communicate to make it happen. And that is at it should be – the architect is the technical expert, but the business people know the business, and change is only worthwhile if it has business value.

“Define their APIs”. Good APIs are always designed. They never evolve by themselves. I know this because I have created some of my own, and I have listened to presentations from the people that designed the APIs for Java and .NET (probably the 2 biggest APIs around). Characteristics of a good API are:

• Easy to learn and use, even without documentation (discoverable)
• Hard to misuse
• Sufficiently powerful to satisfy requirements (but not much more)
• Easy to extend
• Appropriate to audience

The APIs within a system are low-level design. They are important to the maintainability of the system, but they can evolve by themselves, using techniques such as Test-Driven Design (TDD). Their audience is small and specialized. The architect has little business dictating this level of design, although they should certainly dictate which external APIs the developers will be making use of.

The APIs on the edges of a system are at a higher level. TDD (and use of design patterns) makes for bad “system” APIs, because they are not discoverable or appropriate to their audience. To use such an API, you typically need to create a few classes (maybe Strategies), or even implement an interface, and only then can you call a method which will do what you want. This is where the architect needs to step in and (writing code if necessary) ensure that the system can expose itself to the outside world in a proper way. There is clear business value in API design at this level.