Microservices...Complexity

This has been thought many times by Leads/Architects when they begin to think of the product's architecture long before dev folks begin to write code. This could be one of the important decision to make because a simple product could become overly complex OR complex product disguised as simple in beginning could become pain for team(s) to catch up with growing requirements and BUG fixes.

Microservices architecture is indeed a complex thing to manage given the amount of thought process put into planning/designing those tiny/thin services, setting up CI/CD pipeline, separate git repositories.. etc. However if done carefully its pretty much manageable.

It is a distributed architecture and every distributed architecture brings with these concerns

• Security, SSO, Authorization
• Logging
• Service endpoint abstraction - Amount of information sent back to client
• Availability, SLA for each service
• Performance
• Transaction Management - Really you want to look into 2 Phase commit, Its pretty old actually and does not suit well when there are different kinds of DBs used ?
• API Orchestration - Synchronous or Asynchronous communication, Event based

It is not just developing/setting up those tiny/thin services which do not share databases. It's a lot more than that. Many folks underestimate the characteristics of Microservices and begin to plan projects with half knowledge which burns everyone hands later on. One of the most important prerequisite before starting to think of using Microservices is -

• DDD (Domain driven design)
  Its a set of practices and methodology that enables a team to break complex product into more manageable logical pieces which would be much close to real world.

  When your project is close to real world you do not need to use those additional neurons in your brain to map the discussions to your software after a conversation with BA. That's because your software world is the real world so you and your devs save some time/energy there.

• Microservices patterns
  Many concerns which are straight forward to manage in a monolithic (a single solution based project, sharing databases, old protocols/formats) might not be easy thing to chew when stuff gets distributed across tiers. Microservices patterns helps to solve such concerns.

  Chris Richardson does his best to explain these patterns and concerns they solve in this website Microservices and Patterns

Microservices requires to develop a thought process inorder to take right and efficient decisions which can be done if you are well aware of DDD and Microservices patterns.
Explaining everything about Microservices is beyond the scope of this blog and deserves a book of its own. However i have tried best to show the complexity behind microservices which are easily manageable if you know that you are doing right.

Should you be using microservices ?

Amazon, Facebook, Netflix, Google.... and many more companies have started to go service oriented more than a decade ago. So if they do why not us ?

Well.. thats a pretty vague comment to say "YES" unless its backed by a good analysis and research.

An important thing to remember is that things can be changed in future, Your app or services should be moldable, So it depends on how coupled your components/services are, how much abstract your interfaces are ?

These are few items which you should think in depth at the time of this research and decision making

1. Domain
2. Depth of the product/project
3. Integration with other inhouse/3rd party products and services
4. Complexity of the project, number of teams or people involved
5. Release methodology, Versioning, SDLC, CI/CD

Let's go above points one by one

(1). Domain

Having a good knowledge of your domain OR subdomain (a part of the organisation's larger domain) will help you to forecast how future requirements will change existing features and design of the app, You can ask right questions to BA and stakeholders if you know your domain well. Because no one from stakeholders cares about what future requirements/client demands will at present, when they come OR requirement changes its just as simple as to hand over these changelist to engg team and expect it to be completed within the given deadline. If a careful thought is not given over this, then it would create a mess over the time and eventually lead to Big ball of Mud.
If you know your domain, you can create your app or service more modular from design point of view in very beginning it self. Even if requirement changes the impact wont be much because there wont be any hardcode dependencies.

If you remember that we discussed DDD as an important prerequisite, You should be able to answer one of the most important question in microservices

How many number of Bounded Contexts should a subdomain have?
Viz. Does this new feature belong to a new service OR it can be setup in one of my existing services ?

(2). Depth of the product

All points discussed as part of #1 applies here as well as it is.
Your product would be a subdomain or a piece of subdomain.. viz a BoundedContext. One benefit of knowing your product is that you can actively participate during feature planning and create effective features/stories. You can effectively scope your feature/stories with right DoD items and make your Scrum Master and PO happy ;)

(3). Integration with other inhouse/3rd party products and services

You can create effective outbound ports and adapters Viz.. interfaces (refer Hexagonal Architecture). You can structure your project well if you know your consumers well and which architectural design you would like to follow for your project.

• This would require you to think and create an effective API endpoint and responses by applying appropriate REST principles
• You will need to think of concerns we discussed before.

(4). Complexity of the project, number of teams or people involved

Well you cannot attribute a project to be complex only if it makes use of some unique or fancy algorithm but a large project can be overly complex too. Initially it might not seem so, disguised like a simple service if you do not think of above points then it can grow overly complex.
If you know the parts of your products which are complex and need more CPU cycles, they can be extracted to cloud functions (Serverless arhcitecture), OR you can have a dedicated Server for such processing.
Large project means more people and there is a good chance of people stepping onto others feet. Collaboration becomes more difficult if these people spans different teams and teams spanning different locations.
The smaller the project better its manageable, however boundaries should be defined very well

(5). Release methodology, Versioning, SDLC, CI/CD

If your project is to be released frequently, you do not want to request QAs test those part of your project which does not have any logical relation to feature developed.. It might seem so from outside, But from inside they might have some dependency on a class which have been refactored in this release, So every feature which depends on that class needs to be sanity tested.

The point is over the time few of such classes become concrete dependencies and this bond only grows strong over the time. Even if such pieces are extracted out and versioned you cannot use different versions for different modules in same solution/project. Long point short... You cannot perform incremental refactoring (Means that pointing only essential modules which are relative to this release to use newer version of refactored class and letting others to keep on using old one, If they were separate services they can keep on using older version of that lib)
So you might think of not having all modules as part of same solution...... one day. With Large projects it increases the build/deploy time and it only increases. A small change in Module X will trigger build and deployment of your entire service and folks testing Module Y will not be happy with that commit.

So after deeply thinking of these 5 points above, you might now be well aware of whether your new product needs microservice architecture ?

If not right now then you can predict it would, sometime in future because you know your domain/subdomain very well.. and i appreciate that :). The important point to note right now is that you keep on developing your solution more modular and keep your interfaces abstract both internal and external, So that refactoring in future becomes easier.