Load Balancer - 101

Okay, Now What's a Load-Balancer?? 🙄

As the name suggest it has something to do with balancing the load, but here in backend system design architecture load generally refers to traffic or amount of incoming requests 📶

So yeah, going by the name it manages traffic to servers so that they may not get overloaded by traffic ⚖️

Imagine Facebook or Instagram, such a huge giant they are aren't they now let us assume all of their traffic and backend process is handled by a single server, not even a super computer can do such wonders 😂
So definitely they must have multiple server spread across the globe 🌍 now how to decide which request goes to which server 🤨 And how to make sure that a particular server is not stormed by lot of requests, while other servers sits idle and smile at each other 😆

This is the time when load balancer comes for rescue🦸
It does the job of deciding which request goes to which server, and thus solving the above mentioned problems ✔️

Let's see this in detail 🕵️

When we talk about a server its not a single server, actually there are lot of server sitting behind load balancer, Load Balancer is a device (hardware) or may be a software (ex: Ngnix)
But when software is avalible then who will spend money on hardware 💰
So a software load balancer is widely used 🎢

In the above image servers could be web server or application server, and server may further communicate with database etc etc

A load balancer acts as a traffic cop 👮 sitting in front of servers and routing clients across all servers with equal load 📶 This ensures that no single server have too much load which can lead to degradation of service or degradation of server's health 👨‍⚕️

• A load balancer divides load uniformly across all servers, in the above scenario every server would receive 25% of the load as there are 4 servers. ♾️
• When client resolves URL into an IP address through DNS Lookup it basically gives the Load Balancer's IP not server's IP.
• Requests are measured in QPS (Query per second)
• This Adds extra security 🔐 as servers are not publicly exposed and remains in a private network. If a client tries to directly access servers it cant do so without going through load balancer.

Load balancing Algorithms 🍖

The algorithm in a particular load balancer decide which server will be receiving the next incoming request

The Algorithms are as follow 👇

1. Round Robin
2. weighted Round Robin
3. Least Connections
4. Least Response Time
5. IP Hash

Lets take a look at all of them in detail 🕵️‍♂️

Round Robin ⭕

Requests are sequentially sent to 1st, 2nd 3rd,.... till last server and then next request is sent back to the first server and this process is continues in a circular manner 🔁

In the above case requests will be sent in this manner:

1. S1
2. S2
3. S3
4. S4
5. S1
6. S2
7. S3

And So on...

Weighted Round Robin 🏋️

One Drawback in Round Robin is that it assume that all the servers have same computational power and there's no way we can assign more requests to the one with more power and less request to the one with less power. 🚦

To counter this we assign weights to each server and based on that weight requests are redirected 💪

Least Connections Algo ↔️

Request are sent to server with least number of open connections ⚖️

In the above representation at this particular instance S3 has the least number of connections so the next incoming request will be received by server S3 according to Least Connection Algo

Least Response Time Algo 🔭

Request are sent to the server with fewest active connection and lowest average response time ↔️

Here the avg response time is calculated by multiplying number of connections and response time

In the first case S2 and S3 both have same avg response time so in that case either of S2 or S3 will receive the request.

But in case 2 S3 has the least avg response time hence it will receive the request

IP Hashing Algo #️⃣

Here IP of the client is used to determine the server to which request has to be redirected 🥢

An high level explanation would be:

Some logic is written in a function and an IP address is passed to that function, based on that logic which resides inside the function it return the server to which request has to be redirected for that particular IP address 👩🏻‍🔧

Placement of Load Balancers 🛤️

Based on certain criteria Load balancer can be placed at Layer 4 or Layer 7 in the OSI Model of a product 🧘‍♂️

An OSI Model consist of 7 layers which are 👇

Its Similar to layers in a Cake 🥮 😁

Layer 4 Load Balancers:

The Layer 4 Load balancer are placed on the 4th Layer of OSI model i.e Transport Layer

They don't have access to data and decision to redirect request to a server is purely done on the basis on data.

Layer 7 Load Balancers:

Layer 7 Load Balancer are placed on Application layer and have access to incoming as well outgoing request and data and decision is made up on its basis, Layer 7 LB is very very much complex :/

At Last an high level architecture of load balancer can be depicted as:

Happy Load Balancing 🚀