sha254

Posted on Jan 20

Proxy Server in Go

#go #webdev

A proxy server is a system or router that provides a gateway between users and the internet. It helps prevent cyber attackers from entering a private network. There are two types of proxy servers Forward and Reverse .

Reverse Proxy

Reverse proxy servers are installed before the internal servers. Before connection to your internal server, the reverse proxy intercepts all requests from the client before they reach the internal server.

They are generally used to improve security and performance.

Forward Proxy

Forward proxy server are installed between the internal client and external servers. They are meant to intercept request made by internal clients and make requests on behalf of the clients to the external server. It acts on behalf of the client. They are generally used for security reasons.

In this article we are going to implement a reverse proxy server in go

To access the code in this articles go to github.

Setting Up Our Project

First we need to create all the required folder and files.

mkdir -p cmd internal/configs internal/server settings

cmd : It will contain entrypoint of our application.
interna/configs: Will hold code to load configurations values from the yaml file
internal/server : Will hold code to run the HTTP proxy server.
settings : Will contain configuration yaml file for our project.

Let's create all the files required for this project

touch Makefile settings/config.yaml cmd/main.go internal/configs/config.go internal/server/proxy_handler.go internal/server/healthcheck.go internal/server/server.go

Makefile: Allow us to easy run multiple bash commands at once
config.yaml: Hold reverse proxy configurations
main.go: Entrypoint of our program
config.go : Hold code that loads configurations
proxy_handlers.go: Contain functionthat create new proxy and proxy handler
healtcheck.go: Simple http handler to check if our server is running or not.
server.go: It will be used to create and run HTTP server.

Run HTTP Server

We will run the demo http servers in docker containers. Open the Makefile and pasted the following code that runs and stops the the three servers on port 9001, 9002, 9002. And also runs the proxy server.

## run: starts demo http services
.PHONY: run-containers
run-containers:
    docker run --rm -d -p 9001:80 --name server1 kennethreitz/httpbin
    docker run --rm -d -p 9002:80 --name server2 kennethreitz/httpbin
    docker run --rm -d -p 9003:80 --name server3 kennethreitz/httpbin

## stop: stops all demo services
.PHONY: stop
stop:
 docker stop server1
 docker stop server2
 docker stop server3

## run: starts demo http services
.PHONY: run-proxy-server
run-proxy-server:
 go run cmd/main.go

Now when you run make run and make stop respectively it will run and stop the docker containers.

Configure reverse proxy server

We will first configure our server using the config.yaml file.
The file needs to describe
name : Name of the resource we are targeting
endpoint : Endpoint the resource will be serving at
destination_url : Where the incoming request at endpoint will be forwarded to.

server:
  host: "localhost"
  listen_port: "8080"

resources:
  - name: Server1
    endpoint: /server1
    destination_url: "http://localhost:9001"
  - name: Server2
    endpoint: /server2
    destination_url: "http://localhost:9002"
  - name: Server3
    endpoint: /server3
    destination_url: "http://localhost:9003"

Loading Settings from config file

The code to load configurations will be in internal/config/config.go

The code uses Viper to load configuration files in the application.

package configs

import (
 "fmt"
 "strings"
 "github.com/spf13/viper"
)
type resource struct {
 Name            string
 Endpoint        string
 Destination_URL string
}
type configuration struct {
 Server struct {
  Host        string
  Listen_port string
 }
 Resources []resource
}
var Config *configuration
func NewConfiguration() (*configuration, error) {
 viper.AddConfigPath("settings")
 viper.SetConfigName("config")
 viper.SetConfigType("yaml")
 viper.AutomaticEnv()
 viper.SetEnvKeyReplacer(strings.NewReplacer(`.`, `_`))
 err := viper.ReadInConfig()
 if err != nil {
  return nil, fmt.Errorf("error loading config file: %s", err)
 }
 err = viper.Unmarshal(&Config)
 if err != nil {
  return nil, fmt.Errorf("error reading config file: %s", err)
 }
 return Config, nil
}

Creating a proxy and a proxy handler

In file internal/server/proxy_handler.go paste the following code

package server

import (
    "fmt"
    "net/http"
    "net/http/httputil"
    "net/url"
    "strings"
    "time"
)

func NewProxy(target *url.URL) *httputil.ReverseProxy {
    proxy := httputil.NewSingleHostReverseProxy(target)
    return proxy
}

func ProxyRequestHandler(proxy *httputil.ReverseProxy, url *url.URL, endpoint string) func(http.ResponseWriter, *http.Request) {
    return func(w http.ResponseWriter, r *http.Request) {
        fmt.Printf("[ PROXY SERVER ] Request received at %s at %s\n", r.URL, time.Now().UTC())
        // Update the headers to allow for SSL redirection
        r.URL.Host = url.Host
        r.URL.Scheme = url.Scheme
        r.Header.Set("X-Forwarded-Host", r.Header.Get("Host"))
        r.Host = url.Host

        // trim reverseProxyRouterPrefix
        path := r.URL.Path
        r.URL.Path = strings.TrimLeft(path, endpoint)

        // Note that ServeHttp is non blocking and uses a go routine under the hood
        fmt.Printf("[ PROXY SERVER ]Proxying request to %s at %s\n", r.URL, time.Now().UTC())
        proxy.ServeHTTP(w, r)
    }
}

The file has two functions NewProxy and ProxyRequestHandler

NewProxy: Function takes an urland returns a new http reverse proxy

ProxyRequestHandler : takes the proxy create by NewProxy, destination URL and endpoint and returns a http handler.

Creating HTTP server for our proxy

We will create the Run function that we called in main.go.
In the code snippet, we create the Run function which

Loads the configurations by calling NewConfiguration function from our config package.
Create a new server using net/http package.
Register route called /ping to call http handler ping
Iterate through the configuration resources describing the different endpoints and register into our router
We finally start the server

package server

import (
    "fmt"
    "net/http"
    "net/url"
    "reverse-proxy-learn/internal/configs"
)

// Run start the server on defined port
func Run() error {
    // load configurations from config file
    config, err := configs.NewConfiguration()
    if err != nil {
        return fmt.Errorf("could not load configuration: %v", err)
    }

    // Creates a new router
    mux := http.NewServeMux()

    // register health check endpoint
    mux.HandleFunc("/ping", ping)

    // Iterating through the configuration resource and registering them
    // into the router.
    for _, resource := range config.Resources {
        url, _ := url.Parse(resource.Destination_URL)
        proxy := NewProxy(url)
        mux.HandleFunc(resource.Endpoint, ProxyRequestHandler(proxy, url, resource.Endpoint))
    }

    // Running proxy server
    if err := http.ListenAndServe(config.Server.Host+":"+config.Server.Listen_port, mux); err != nil {
        return fmt.Errorf("could not start the server: %v", err)
    }

    return nil
}

We will also need to implement the /ping route handler. Paste the snippet below in the file internal/server/healthcheck.go

package server

import "net/http"

// ping returns a "pong" message
func ping(w http.ResponseWriter, r *http.Request) {
    w.Write([]byte("pong"))
}

Entrypoint for the Application

Inside cmd/main.go paste the following

package main

import (
    "log"

    "reverse-proxy-learn/internal/server"
)

func main() {
    if err := server.Run(); err != nil {
        log.Fatalf("could not start the server: %v", err)
    }
}

We can finally run our program buy calling server.Run() and handling any error if returned.

Running and testing Proxy

To run the program we need to first run our destination servers.

make run-containers

Once destnation servers are running run the proxy server

make run-proxy-server

which run the code.

To test that the proxy works.
Run the snippet below to attempt to connect to server 1

curl -I http://localhost:8080/server1

The response should be in this format

HTTP/1.1 200 OK
Access-Control-Allow-Credentials: true
Access-Control-Allow-Origin: *
Content-Length: 9593
Content-Type: text/html; charset=utf-8
Date: Sat, 20 Jan 2024 05:24:51 GMT
Server: gunicorn/19.9.0

We can verify that proxy as intended by looking at the servers logs

[ PROXY SERVER ] Request received at /server1 at 2024-01-20 05:24:30.616616225 +0000 UTC
[ PROXY SERVER ]Proxying request to http://localhost:9001 at 2024-01-20 05:24:30.616692726 +0000 UTC
[ PROXY SERVER ] Request received at /server1 at 2024-01-20 05:24:51.927329445 +0000 UTC
[ PROXY SERVER ]Proxying request to http://localhost:9001 at 2024-01-20 05:24:51.927399846 +0000 UTC