The world is composed of complex systems. Engineering projects, financial portfolios, software platforms and healthcare processes to name but a few. No discipline is left untouched by complexity.
Hugely complicated systems can function flawlessly for decades, but more often than not they collapse under the weight of their intricacies before even seeing the light of day.
The chance of success or failure is broadly determined by how the system came into being:
“A complex system that works is invariably found to have evolved from a simple system that worked. A complex system designed from scratch never works and cannot be patched up to make it work. You have to start over with a working simple system” — Gall’s Law
Designing a system from scratch
Complex systems are full of variables and interdependencies that need to be arranged in just the right way to function well. A tangled web of interactions, regardless of whether they’re human or computer, introduces uncertainty. It is impossible to predict how such a system will behave ahead of time. It’s likely to fail in ways you’d never dreamt of — you cannot plan for every eventuality.
Attempting to build a complex system from scratch is likely to produce nothing more than an expensive, demoralizing, failure.
Start simple and evolve
Gall’s law states that a complex system cannot be built from scratch but only grown from simpler working systems.
This is an evolutionary viewpoint. By starting small and gradually growing the behaviour of a system, we’re able to meet each challenge that stands in its way, many of which we could not have predicted beforehand. Each test filters out the wrong behaviours of the system and evolves it in the most useful direction.
It is also possible to compose a number of existing, working, systems into something greater than its parts. Think of the UNIX philosophy — small, simple, modules that are able to be composed in a myriad of ways.
Strategies for safe growth
Release early, release often
As popularized by Eric S. Raymond in his 1997 essay The Cathedral and the Bazaar, by releasing early and often, we continually test the system against its users and are able to adapt it when the problem is simple and easy to change. This philosophy attempts to lower the risk of creating software that no one will use.
Automated Testing
Gall makes it very clear that the simple system we’re building upon must be a working system. Clearly you cannot build a more complex system on top of one that already doesn’t work.
Automated tests ensure that we do not compromise the simpler system as we evolve it.
Strangler Fig Pattern
The second system effect is “the tendency of small, elegant, and successful systems to be succeeded by over-engineered, bloated systems, due to inflated expectations and overconfidence”. It is building a complex system from scratch.
Rather than starting over, through use of the strangler fig pattern we can move or replace small parts of the system whilst keeping the rest of it constant. This introduces much less risk and gives us much more control.
Further Reading
Systemantics: How Systems Really Work and How They Fail, John Gall
A final word
This post was written as part of a series on laws of software development for #PragProWriMo 2021 run by the The Pragmatic Programmers.
Top comments (0)