Kenta Takeuchi

Posted on Jan 1, 2022 • Updated on Jan 18, 2022 • Originally published at bmf-tech.com

Implement a load balancer in Golang

#go #loadbalancer #roundrobin

Overview

This article is a translation of Golangでロードバランサーを実装する

This article is the 24th day article of Makuake Advent Calendar 2021.

It is a story of trying to make your own load balancer with Golang that distributes the load with round robin.

What is a load balancer

A load balancer is a server that has a load balancing function that distributes requests to multiple servers.

It is a kind of reverse proxy that increases the availability of services.

There are two main types of load balancers.

An L7 load balancer that load balances at the application layer and an L4 load balancer that load balances at the transport layer.

In addition to load balancing, the load balancer has the functions of persistence (session maintenance) and health check.

Type of load balancing

There are different types of load balancing, static and dynamic.

A typical static method is Round Robin, which distributes requests evenly.

A typical dynamic method is Least Connection, which distributes requests to the server with the least number of unprocessed requests.

Types of persistence

Persistence is a function for maintaining a session between multiple servers to which the load balancer is distributed.

There are two main methods, Source address affinity persistence and Cookie persistence.

Source address affinity persistence is a method to fix the destination server by looking at the source IP address.

Cookie persistence is a method that issues a cookie to maintain a session, looks at the cookie, and fixes the server to which it is distributed.

Health check type

Health check is a function that the load balancer checks the operating status of the distribution destination server.

An active health check method that checks the health of the load balancer to the server to which it is distributed, and a method that monitors the response to requests from clients.

Active checks can be classified into L3 check, L4 check, and L7 check depending on the protocol used.

Implementation

Implement the L4 load balancer as a package.

The type of load balancing is round robin, and the health check corresponds to active check and passive check respectively.

Persistence does not support.

The code implemented this time can be found at github.com/bmf-san/godon.

Implement a reverse proxy

A load balancer is a type of reverse proxy. Let's start by implementing a simple reverse proxy.

Golang can be easily implemented by using httputil.

package godon

import (
    "log"
    "net/http"
    "net/http/httputil"
)


func Serve() {
    director := func(request *http.Request) {
        request.URL.Scheme = "http"
        request.URL.Host = ":8081"
    }

    rp := &httputil.ReverseProxy{
        Director: director,
    }

    s := http.Server{
        Addr:    ":8080",
        Handler: rp,
    }

    if err := s.ListenAndServe(); err != nil {
        log.Fatal(err.Error())
    }
}

I won't explain it here, but I think it's a good idea to read pkg.go.dev/net/http/httputil#ReverseProxy carefully.

Implementation of Config

Since it is a simple load balancer, it does not have complicated settings, but I will implement a setting function that reads the settings from json.

{
    "proxy": {
        "port": "8080"
    },
    "backends": [
        {
            "url": "http://localhost:8081/"
        },
        {
            "url": "http://localhost:8082/"
        },
        {
            "url": "http://localhost:8083/"
        },
        {
            "url": "http://localhost:8084/"
        }
    ]
}

// ...

// Config is a configuration.
type Config struct {
    Proxy    Proxy     `json:"proxy"`
    Backends []Backend `json:"backends"`
}

// Proxy is a reverse proxy, and means load balancer.
type Proxy struct {
    Port string `json:"port"`
}

// Backend is servers which load balancer is transferred.
type Backend struct {
    URL    string `json:"url"`
    IsDead bool
    mu     sync.RWMutex
}

var cfg Config

// Serve serves a loadbalancer.
func Serve() {
    // ...

    data, err := ioutil.ReadFile("./config.json")
    if err != nil {
        log.Fatal(err.Error())
    }
    json.Unmarshal(data, &cfg)

    // ...
}

Implementation of round robin

Next, we will implement round robin.

All you have to do is distribute the requests evenly to the backend server, and implement it regardless of whether the backend server is alive or dead.

// ...

var mu sync.Mutex
var idx int = 0

// lbHandler is a handler for loadbalancing
func lbHandler(w http.ResponseWriter, r *http.Request) {
    maxLen := len(cfg.Backends)
    // Round Robin
    mu.Lock()
    currentBackend := cfg.Backends[idx%maxLen]
    targetURL, err := url.Parse(cfg.Backends[idx%maxLen].URL)
    if err != nil {
        log.Fatal(err.Error())
    }
    idx++
    mu.Unlock()
    reverseProxy := httputil.NewSingleHostReverseProxy(targetURL)
    reverseProxy.ServeHTTP(w, r)
}

// ...

var cfg Config

// Serve serves a loadbalancer.
func Serve() {
    data, err := ioutil.ReadFile("./config.json")
    if err != nil {
        log.Fatal(err.Error())
    }
    json.Unmarshal(data, &cfg)

    s := http.Server{
        Addr:    ":" + cfg.Proxy.Port,
        Handler: http.HandlerFunc(lbHandler),
    }
    if err = s.ListenAndServe(); err != nil {
        log.Fatal(err.Error())
    }
}

The reason for using sync.Mutex is to avoid race conditions caused by multiple Goroutines accessing variables.

You can check the race condition by removing sync.Mutex, starting the server withgo run -race server.go, and requesting from multiple terminals at the same time.

Implementation of active check

In the implementation so far, the load balancer has the logic to forward the request even to the abnormal backend.

In a real use case, you don't want the request to be forwarded to the anomalous backend, so detect the anomalous backend and try to deviate from the destination.

// Backend is servers which load balancer is transferred.
type Backend struct {
    URL    string `json:"url"`
    IsDead bool
    mu     sync.RWMutex
}

// SetDead updates the value of IsDead in Backend.
func (backend *Backend) SetDead(b bool) {
    backend.mu.Lock()
    backend.IsDead = b
    backend.mu.Unlock()
}

// GetIsDead returns the value of IsDead in Backend.
func (backend *Backend) GetIsDead() bool {
    backend.mu.RLock()
    isAlive := backend.IsDead
    backend.mu.RUnlock()
    return isAlive
}

var mu sync.Mutex
var idx int = 0

// lbHandler is a handler for loadbalancing
func lbHandler(w http.ResponseWriter, r *http.Request) {
    maxLen := len(cfg.Backends)
    // Round Robin
    mu.Lock()
    currentBackend := cfg.Backends[idx%maxLen]
    if currentBackend.GetIsDead() {
        idx++
    }
    targetURL, err := url.Parse(cfg.Backends[idx%maxLen].URL)
    if err != nil {
        log.Fatal(err.Error())
    }
    idx++
    mu.Unlock()
    reverseProxy := httputil.NewSingleHostReverseProxy(targetURL)
    reverseProxy.ErrorHandler = func(w http.ResponseWriter, r *http.Request, e error) {
        // NOTE: It is better to implement retry.
        log.Printf("%v is dead.", targetURL)
        currentBackend.SetDead(true)
        lbHandler(w, r)
    }
    reverseProxy.ServeHTTP(w, r)
}

var cfg Config

// Serve serves a loadbalancer.
func Serve() {
    data, err := ioutil.ReadFile("./config.json")
    if err != nil {
        log.Fatal(err.Error())
    }
    json.Unmarshal(data, &cfg)

    s := http.Server{
        Addr:    ":" + cfg.Proxy.Port,
        Handler: http.HandlerFunc(lbHandler),
    }
    if err = s.ListenAndServe(); err != nil {
        log.Fatal(err.Error())
    }
}

Implements a ErrorHandler that is called when the load balancer detects an error when forwarding a request to the backend.

In ErrorHandler, the backend that does not return a response normally is flagged so that the load balancer forwards the request again.

The load balancer is adjusting its logic so that it does not forward requests to the flagged backend.

Implementation of passive check

Finally, we will implement a passive check.

Passive checks only monitor the response of the backend server at intervals.

The backend where the anomaly is detected is flagged as it was during the active check.

Below is all the code that has implemented the passive check.

package godon

import (
    "encoding/json"
    "io/ioutil"
    "log"
    "net"
    "net/http"
    "net/http/httputil"
    "net/url"
    "sync"
    "time"
)

// Config is a configuration.
type Config struct {
    Proxy    Proxy     `json:"proxy"`
    Backends []Backend `json:"backends"`
}

// Proxy is a reverse proxy, and means load balancer.
type Proxy struct {
    Port string `json:"port"`
}

// Backend is servers which load balancer is transferred.
type Backend struct {
    URL    string `json:"url"`
    IsDead bool
    mu     sync.RWMutex
}

// SetDead updates the value of IsDead in Backend.
func (backend *Backend) SetDead(b bool) {
    backend.mu.Lock()
    backend.IsDead = b
    backend.mu.Unlock()
}

// GetIsDead returns the value of IsDead in Backend.
func (backend *Backend) GetIsDead() bool {
    backend.mu.RLock()
    isAlive := backend.IsDead
    backend.mu.RUnlock()
    return isAlive
}

var mu sync.Mutex
var idx int = 0

// lbHandler is a handler for loadbalancing
func lbHandler(w http.ResponseWriter, r *http.Request) {
    maxLen := len(cfg.Backends)
    // Round Robin
    mu.Lock()
    currentBackend := cfg.Backends[idx%maxLen]
    if currentBackend.GetIsDead() {
        idx++
    }
    targetURL, err := url.Parse(cfg.Backends[idx%maxLen].URL)
    if err != nil {
        log.Fatal(err.Error())
    }
    idx++
    mu.Unlock()
    reverseProxy := httputil.NewSingleHostReverseProxy(targetURL)
    reverseProxy.ErrorHandler = func(w http.ResponseWriter, r *http.Request, e error) {
        // NOTE: It is better to implement retry.
        log.Printf("%v is dead.", targetURL)
        currentBackend.SetDead(true)
        lbHandler(w, r)
    }
    reverseProxy.ServeHTTP(w, r)
}

// pingBackend checks if the backend is alive.
func isAlive(url *url.URL) bool {
    conn, err := net.DialTimeout("tcp", url.Host, time.Minute*1)
    if err != nil {
        log.Printf("Unreachable to %v, error:", url.Host, err.Error())
        return false
    }
    defer conn.Close()
    return true
}

// healthCheck is a function for healthcheck
func healthCheck() {
    t := time.NewTicker(time.Minute * 1)
    for {
        select {
        case <-t.C:
            for _, backend := range cfg.Backends {
                pingURL, err := url.Parse(backend.URL)
                if err != nil {
                    log.Fatal(err.Error())
                }
                isAlive := isAlive(pingURL)
                backend.SetDead(!isAlive)
                msg := "ok"
                if !isAlive {
                    msg = "dead"
                }
                log.Printf("%v checked %v by healthcheck", backend.URL, msg)
            }
        }
    }

}

var cfg Config

// Serve serves a loadbalancer.
func Serve() {
    data, err := ioutil.ReadFile("./config.json")
    if err != nil {
        log.Fatal(err.Error())
    }
    json.Unmarshal(data, &cfg)

    go healthCheck()

    s := http.Server{
        Addr:    ":" + cfg.Proxy.Port,
        Handler: http.HandlerFunc(lbHandler),
    }
    if err = s.ListenAndServe(); err != nil {
        log.Fatal(err.Error())
    }
}

Impressions

I haven't been able to implement retries or support persistence, but I think you've found that Golang makes it relatively easy to implement a load balancer.

References

Top comments (2)

Jaroslav Rakhmatoullin • Jan 9 '23

Najsu. Gooda usu ofa composition. I rike the indexu by mod tricku. A rittre ineffectivu in term ofa CPU anda Memoly, but good enough as exampru! Senk u, nii-chan!

Lakin Mohapatra • Jan 14 '22

This is really cool

DEV Community