Most if not all programmers nowadays are familiar with the concept of clean code. In short, the clean code principle means a development style that focuses on code that is easy to write, understand and maintain. Yet, it seems that often these same principles are not applied to the overall design.
Any fool can write code that a computer can understand. Good programmers write code that humans can understand.
- Martin Fowler
Still, even if a code base has clear naming practices, meaningful comments and the code in function-level is understandable and clean, it is still possible that the overall design is hard to comprehend and the project is hard to maintain. This is often because the same clean coding methodology that is applied to code in small is not applied to the overall design. The design is over-engineered.
Sometimes the code base looks as if the author read "the Gang of Four" design pattern book and tried to apply every single one of them in a single code base. There are just facades, decorators, adapters, observers and other design patterns on top of one another and the overall design is nearly impossible to grasp. Other times it seems that the author had an impulse to reinvent every wheel there is. Own containers. Own locking primitives. Own algorithms. Everything written from scratch.
Overly complex, over-engineered code can be as hard to understand and maintain as badly written spaghetti code. Of course, sometimes projects really are complex and require abstractions and architecture that is challenging to understand. There also certainly are situations where implementing your own algorithms or containers is the right thing to do. The important thing is that there should be a clear reason why that additional complexity is necessary. The design should never be complex just for the sake of complexity. In fact, contrived complexity, forced use of overcomplicated design when simple would suffice, is one of the common code smells described by Martin Fowler.
Projects may end up overly complex for a number of reasons. The design might be clean and simple in the beginning, but become muddy and complex when new features and other changes that not quite fit the architecture are glued into it without refactoring. Other projects might be too complex from the get-go when the design tries to accommodate all imaginable changes that might happen in the future.
Thing is, designing a good solid architecture is not easy. In real-world projects requirements change along the way and new features are introduced that were not known in the beginning. Also when the project matures, other fixes, tweaks and updates are applied along the way. On top of that, project schedule often adds pressure to get things done quickly. The design should be flexible enough to allow all these changes with reasonable effort while still being as simple as reasonably possible. And even without all these external changes it is hard the get the design right on the first try. It usually takes some iterations and refinements to converge to the final design
Unfortunately there is no silver bullet that always gives an optimal architecture. But applying the same clean coding principles that work on function and class level to architecture is a good way to avoid muddy and over-engineered design. Probably the most important advice is to keep things simple. Add configurations and use design patterns only when really needed. Create abstractions only when they contribute positively to the design by removing duplication or decoupling modules for instance. Don't try to predict future. Design the system using requirements that are actually known today.
When the future does come and the design needs to adapt, it is important that the system is easy to change. Often new features require refactoring the existing design or otherwise they are just glued in which decays the architecture. Soon even simple changes can break the system unexpectedly from multiple places and subtle bugs are introduced. Changes start to take longer and longer to implement because the design is too fragile.
The key to make the design easy to change is to make it easy to test. When the functionality can be verified easily and automatically, developers can make refactorings without being scared of breaking things. When the system is easy to test, it is easier to refactor continuously and thus keep both code in small as well as the larger design clean. Small imperfections and problems tend to grow so deal with them immediately.
One broken window, left unrepaired for any substantial length of time, instills in the inhabitants of the building a sense of abandonment—a sense that the powers that be don’t care about the building. So another window gets broken. People start littering. Graffiti appears. Serious structural damage begins. In a relatively short space of time, the building becomes damaged beyond the owner’s desire to fix it, and the sense of abandonment becomes reality.
- Andy Hunt, The Pragmatic Programmer
Don't live with broken windows. Make sure both the code and the larger architecture stays simple and clean. Refactor continuously to accommodate changes, and add abstraction and complexity only when it is justifiable and contributes positively to the design. Don't over-engineer.
Originally posted on my personal blog
Top comments (2)
Interesting read, thanks Sami!
I haven't yet experienced this kind of situation. However i'm very familiar with the opposite. :)
I've heard a more than a few time the mantra "don't over-engineer" as an excuse to not do design at all. The thought is usually "in which class could i cram this piece of code" which imo usually indicates that there isn't enough abstraction. Also object have too many responsibilities, know and do too much.
Yes, but beware of primitive obsession. If it's not in the business domain or isn't a clear or well known pattern then i suppose it creates confusion.
Someone smart has said that don't design to patterns, but refactor into patterns.
I'd be curious if you could show me an example where things are clearly over-engineered?
First of all, thanks for reading and commenting! Very good observations indeed.
I think fixing the primitive obsession is a prime example of a situation where the added abstraction and complexity (new classes and types, for instance) have a significant positive impact to the overall design. But as you remarked, it is important not to misinterpret "don't over-engineer" with don't design at all.
The sentence about overuse of design patterns is a bit exaggerative and I am by no means against them in general. In fact they can convey a lot of information by providing a common vocabulary (if I see that something is for example an observer or a factory I immediately have a pretty clear idea what it is). Still, it is important to avoid "when you have a hammer, everything looks like a nail" type of situation where same patterns are applied to all problems whether they are a good fit or not.
I have encountered this mainly in some older C++ codebases that I have worked with. Though, the opposite "no design at all" might be even more common as you said. Unfortunately these have been closed source so I cannot directly link to any code.
One concrete situation I have seen is overuse of abstract classes and interfaces where many classes have accompanied abstract class or interface even though it does not provide any value. For instance, internal classes that do not need to be heavily decoupled or only one concrete child class. I suppose they are added "just in case" but could be refactored in when actually needed. Another case is own implementation of common algorithms or containers without documentation or rationale why the standard ones were not used.
I would say that most of the time things are not grossly over-engineered but sometimes you'll notice that it takes extra time to understand some piece of code only to realize later that it should not have been that complex.