Thoughts in the Wilderness

My first rule of good software architecture dealt with ways of making a particular code-base last a long time. The focus of the 2nd rule is different – it assumes that a problem domain will be solved multiple times by different software, or multiple versions of the same software. It suggests ways that we can make each new re-solving of the problem easier than the last.

To review, my 2nd law of good software architecture is:

Keep as much information as possible in an accessible, declarative form. This will eliminate duplication, and enable your software to be discarded and re-written without losing quite as much

This law is all about re-use of information, and describing how a particular problem domain can become better understood, even to the extreme where the “software” is just data.

We’ve all seen the tool-sets for generating entire applications – enter your requirements (mostly just your data structure) using vendor X’s WonderMaker(tm) and lo and behold, out springs an application with handy generated forms for doing wonderful things. As it turns out, those wonderful things are pretty much Create, Read, Update and Delete. Not so useful after all.

Those generic tools do solve particular problems well – but it is usually not the problem we want to solve. Understandably, users demand more than just create, read update and delete – they want to use the software to perform some task that meets their goals.

We can achieve the goal of a tool that generates most of an application, but only once we understand the problem domain well enough. We need to understand the domain, because we need to know what we can generate, and what we need to leave open to extension.

It is always a mistake to design a v1.0 system where logic is executed based on models of application logic. There are plenty of horror stories about the architect who thought he could model the business logic using XML. Don’t be the next one. This post is about evolving your understanding of a particular domain to the point where you can create models with confidence they will work.

That said, even in version 1.0, there are some things we can recognize. The first is that there are at least two easily identified models of the system. From the user’s perspective, there is the model that they understand and interact with. At the other end, there is the database. The important logic of the application sits between the user’s model and the database. This is the origin of the old 3-tiered concept – UI + Application + Database.

What we have to realize is that we can model each of these things in a way that is declarative. In version 1.0, we may not understand the way the user wants to use the UI well enough to do much work in this regard. However, we can certainly model the database in declarative form, and have that model persist after version 1.0.

Modeling the Application layer is the last evolutionary step. You will not reach that point until the core application requirements are stable and well-understood.

For existing products, the process of modeling requires a re-write of some portion of the system. This is unavoidable, because you have to extract information from where it is hidden in the code, and represent it outside of the code. The code will no longer work. The good news is that once you have correctly modeled a part of the system, the model can be extended to capture new types of information, and need not be re-written again.

A re-usable database Model
So what does a re-usable database model look like?

• It treats relationships as a first class concept – they have names, and they have attributes (one-to-many, cascade-delete behavior).
• It describes fields in a rich, descriptive manner. Strings have maximum lengths, phone numbers are represented by a phone-number data type, etc.
• It describes lookups (sets of values that are acceptable for a field)
• It describes roles – how field values come together to represent a particular flavor of record that has meaning to the user. A particular flavor may be extended with additional fields and properties.
• It should be directly and easily accessible to the rest of the code (re-usable).
• It should be able to be transformed into something that the data access layer (or ORM tool) can use directly.
• It should be able to be transformed into an empty database (it is complete).

The most obvious storage form of the model is as XML, because it is very accessible, and because it can represent hierarchical data. Other forms are ok, as long as they meet the above criteria.

Why all of the richness? We want to capture as much information as possible in a single place. This allows us to make use of that information at higher layers of the application, in ways that enhance the user and the developer experience. DRY (Don’t Repeat Yourself) is a powerful architectural technique.

Not all database structures represent the model we wish we had. We may have inherited a database, and it may be a horrible thing to behold. My first law of good software architecture applies – since we are exposing the model directly to the developer, we want it to be the one we wish we had. If the real database is too far from what we want, then we need to take steps to address that inconsistency. To do otherwise is to invite artificial complexity in the application code.

A re-usable UI Model
As mentioned previously, I do not expect that many version 1.0 products have a very good UI model. Still, if we can understand what a UI model looks like, then we can work towards it.

Firstly, a UI model has a relationship with the database model. The relationship is mapped – i.e. there is some automated transformation that can be used to relate a field on the UI back to one or more fields in the database. This is important, because the relationship is what allows us to re-use information defined at the database model (such as rich data types, lookups, maximum field lengths etc). If we’re re-using information, then we are not duplicating it.

A UI model can grow in pieces. First, you can model screens, then larger pieces that describe how various screens fit together. Screen models are the easiest. (Even today, many applications make use of screen models).

Again, XML is a good choice for representing the UI model.

Beware of including layout information in the UI model. That is a different aspect that belongs in a different model. The primary purpose of the UI model is to bring together fields and screens in a way that represents how the user sees them. This may include their likely order on the screen, but should not include their actual co-ordinates.

UI models are re-usable in several ways. Security, Form Design, and Ad-hoc user queries are a few.

Layout of Forms (Views)
Form layout can be defined declaratively, but it is seldom worth the trouble to do that manually. We cannot predict the next evolution of UI well enough to design a representation that is good enough. The best you can probably do is favor form-design tools that save themselves declaratively (for example, XAML).

Your form layout should make use of the UI Model directly (via data binding and control-binding). Otherwise, you are just duplicating yourself. (Control-binding is the technique of having the final appearance of a particular control determined based on metadata. See the screen shots in my Egg UI post for
an example).

Form layouts can be generated. This is a dangerous path, because it can limit your ability to satisfy the needs of the end-user.

Security
It is particularly useful to relate security to the UI model. One reason is that security is highly contextual – whether a user has rights to touch particular data elements can be driven by many factors, including the time of day. Another reason is that users need to understand security in order to effectively define it. The UI model’s shared understanding of the user’s perspective allows a good point of interaction for security.

Ad-hoc user queries
Often, we may want to expose the ability for users to query a database in some way that is fairly dynamic. A UI model that is mapped back to the database model provides a simple way to provide that feature.

See Matt Blodgett’s First Law of Software Development

A development process that involves any amount of tedium will eventually be done poorly or not at all.

I like that. To me, it is yet another argument for DRY (Don’t Repeat Yourself), which I consider to be the most important aspect of long term software quality.

If you are doing DRY, then you are not repeating yourself. Therefore, you are doing the least amount that you can in order to solve the problem. Any tedium is thus inherent in the problem, and could not be avoided.

(Of course, if you find or invent the right tool, you can also mitigate the remaining tedium. For example, using a diagramming tool to draw your database relationships rather than typing them in XML or SQL).

I started using Castle ActiveRecord last week, in an effort to refactor the data access layer of a medium-size application that I have to re-write (or at least rinse, DRY and repeat until it is maintainable).

Firstly, kudos to the Castle team. For the most part, they have made the simple things simple. In case you’re not familiar, Castle ActiveRecord is an API that supports the ActiveRecord data access pattern. Rather than re-invent the wheel, they use the robust (but somewhat complex and under-documented) NHibernate O/R Mapper for their data access.

Anyway, on to the subject at hand – HQL = Hibernate Query Language. I was hesitant to use this at first, because it is after all *text*. I dislike using text-sql in my applications because there is no type-safety, and that limits my ability to change the code. Nevertheless, as part of the refactoring process, I decided to leave some of the SQL there, for now.

I did not want to learn HQL. It seemed a waste – I don’t need to work with multiple database flavors – just with Sql Server. But after many struggles, I eventually did learn it. And I am glad, because I quickly (ok, slowly) realized that HQL addressed my biggest pet peeve regarding normal SQL – it treats relations as first class citizens. This leads to wonderful, easy to read and write queries, almost exactly like I thought they should look (see my previous post for my thoughts on that).

HQL – if you have been avoiding it, consider taking a second look. Its much more than just a way of abstracting the database implementation. It is SQL, the way it should have been.

Just doodling…What’s the most important aspect of long-term-quality software? I’ll define long-term quality software as some piece of software with a lifespan of many years, over which that software can be extended and changed to suit new needs without compromising quality.

Some potential answers, with my best spur-of-the-moment arguments:

• Strong typing – not just variable types, but any sort of type, like a database table. If something changes in the contract (field name changes, or a 1-1 relationship becomes 1-many), then I should be able to make the change and within minutes, know each of the places that are impacted in the code. The justified fear of making changes to a system is driven by the unknown impacts. If I know all of the impacts, then I am in a very strong place.
• DRY (Don’t Repeat Yourself) – If logic is represented in multiple places, then someone will only change it in one, which will automatically create some inconsistency. If you are lucky, then the inconsistency will be noticed quickly. If you are not, then you will only find out later when the damage is done.
• YAGNI (You Ain’t Gonna Need It) – Software is complex. At some point, the complexity becomes too much for us to fit in our minds at one time. The longer we can defer that point in time, the more maintainable (and learnable) the software will be. There are two distinct types of complexity though – inherent complexity (because the problem is complex) and artificial (unnecessary) complexity. By introducing functionality before we know for sure that we need it, we are creating artificial complexity. Thus, we will reach the point of too much complexity before we should have.
• Minimized Coupling – The complexity of software is directly related to how big it is. When we couple things together, we are making something more monolithic, and thus harder to understand. We also cross a line that is difficult to un-cross. (One coupling-point is just the first of many). Minimized coupling is an antidote to complexity.

I think minimized coupling is perhaps the most important, because it has such a direct impact on complexity. I looove DRY though – it is addictive once you try it in earnest. Strong-typing is of limited use without DRY. YAGNI is good advice, although some take it too far.

Are there other candidates?

Traditional OO lore teaches us that objects are things that have both data and behavior. Blindly following this rule can lead us to make poor design choices, especially around what many refer to as “business objects”.

The pattern is that these objects already have data, so we seek to add behavior as well. In this way we can feel happy and content that we have a true “object”, and we are successful OO programmers.

The problem is that adding behavior as a sort of “suffix” to an object is ignoring a more important aspect of objects, which is that they should do one thing, and do it well. Add too many “suffix” behaviors, and pretty soon you can have a tightly coupled bowl of spaghetti.

This is not just theoretical – I have seen it happen, more than once. I’ve even been guilty of it.

So what is the solution? When we have classes that are primarily data, should we resist adding behavior?

My answer is “it depends”. To understand why, we need to take a small detour into API design…

Sometimes, programmers expect things to be a certain, simple way. They do not want to ask a FactoryLocator for an IObjectPersistorFactory, use that to get an IObjectPersistor, and finally tell the IObjectPersistor to Save their object to the database. They just want to write:

myObject.Save()
or
myObject.Load(id)

This ActiveRecord implementation is easy to write and easy to read. In short, it is good because it is a nice API for the client of the object. It has drawbacks (no transaction support, high risk of coupling to database). But in many systems, this API will be sufficient.

So the ActiveRecord “suffix” is mostly ok. What other behaviors can we add? How about validation? The save method should probably validate before it saves, so as to ensure we have good data in the database. How about some initial field values for new objects? And some event driven behavior – let field A be defaulted when field B changes? And we need properties for other objects. MyCustomer.Address.ZipCode works real nice. We can even lazy-load the Address property. Not too hard.

Hmm. Question. If we save the Customer object, should the Address save too? Probably. So we need to add some more code to the Save method for that.

etc. etc.

You get the picture (I hope). You can create a perfectly functional system in this way, but the coupling of all functionality to a single class will make it difficult to change in any substantial way. It will also have poor quality, because we are ignoring several key principles, such as DRY and Open-Closed.

There is only one way in which you can mitigate the problem. Use code-generation to generate your “business object” implementations. This mitigates quality problems substantially (DRY does not apply to generated code). It also forces you to either state some things declaratively (such as required fields), or else move them into their own dedicated area.