There is a lot to be said on the pros and cons of a monolith vs microservice software design (just Google a few). To briefly sum things up; while monolith applications are often still the best choice, organisations are increasingly leaning toward microservices architecture. Many companies recognise the importance of increasing overall engineering efficiency through team/unit autonomy and access to a wider labour market. And most are facing similar challenges in their journey to microservices.
Starting a transformation without a plan imposes significant risks. Many challenges must be tackled beforehand which play a foundational role in the success of adopting a monolith system. These include data input, output, processing, error handling, and the UI, which are usually written once and reused across the whole monolith.
Now, imagine multiplying these entities by the amount of stacks or a number of services we envision to expand into. Missing a common layer and principles across services will harm engineering efficiency by a lot. After all, if engineers are to produce tangible value. we must address common problems before we can put a foot in new water on a scale of 50+ people.
Get ready
At Catawiki, we’ve built a foundation with an app skeleton repo, automation and guidelines to enable teams to consistently deliver new services. Besides that, we've built libraries with the most essential tools helping us to streamline integration. This simple 2-step plan is for many adopters often just the tip of the iceberg. And the hidden part is bigger, further away we are from Cloud Native toolchain.
Organisational-wide standards are vital for the success of the microservices journey. It is best to start early with an internal SDK, OpenAPI specification and infrastructure as code. Nowadays, these problems are solved with a myriad of services, tools and well-established standards. Some monolith code will likely come in handy or become a blueprint for a common library that each app may depend on.
Focus on Data
Most applications are dealing with data and underlying storage. While an application layer is usually scaled by simply multiplying compute workload, a data layer eventually becomes a bottleneck.
Inefficient data structures and data-querying algorithms are the biggest problems we'll have to deal with when scaling an application. Frederick Brooks, the author of The Mythical Man-Month, says:
The programmer's primary weapon in the never-ending battle against slow systems is to change the intramodular structure. Our first response should be to reorganize the modules' data structures.
Microservices can bring in significant costs if the data design is not addressed properly. Replacing in-process code execution with cross-host API calls via extra virtual and physical processes costs user time and is more expensive for businesses to operate. In fact, the efficiency of any service depends on how data is stored and exchanged among services.
As the data architecture becomes so important, we'll have to be smart about how we denormalise and exchange it. Categorise data and its usage, establish data exchange rules and apply the right tools for specific use cases. It is always difficult to change data structures at a later stage, so best to ensure storage scales x10-x100 times and return the right value for the years ahead.
Realising the effort
When everything is moving fast in a growing business, it’s not easy to change overnight. To create a system developed from scratch by five people over five years, a quick estimate suggests the time needed amounts to at least 50.000 hours of engineering work. Of course, rebuilding a monolith into microservices might be less expensive than the original investment, as we may borrow from the old system some code and established business logic.
However, for an established business, a rapid rebuild from scratch is usually not an option as it requires us to stop feature development in the old system to avoid ending up a never-ending cycle of catching up with updates. This means that we have to build a new system and features working alongside the old one, exchanging and duplicating data and features, in order to provide clients with a gradual transition away from the original system..
It's true that an effort like this can cost any company a huge amount of time and resources. Nevertheless, organisational and innovation benefits of modern Service Oriented Architecture (and Microservices specifically) will easily outweigh any benefits of a Monolith system in the long run. To make a transition successful and get buy-in from the company at large, it is important to plan things ,with a clear rationale as to why you’re doing this and an understanding of the goals.
Be organised
An organisation with hundreds of people is a complex structure. After years of slow progress in our journey to Microservices, we've realised a lot of things. Perhaps the most important one is that this challenge is purely in the hands of engineers and as such must be driven by engineers.
To better organise ourselves, we've built a company-wide Engineering Roadmap reflecting all steps of migration, accounting for people we have and/or will be hired this year. And with a clear vision of what this is going to bring us, we've committed to making a huge leap this year toward a better, more efficient and scalable foundation of our unique platform for passionate enthusiasts all around the world.
A few months later down the road, we're extremely satisfied with the results and the progress we've made so far. Although this year has just started, we are already planning for exciting engineering challenges in 2022 and beyond!
Cover photo by Boba Jaglicic on Unsplash.
Top comments (0)