DEV Community

Kazuki Higashiguchi
Kazuki Higashiguchi

Posted on • Updated on

Reverse HTTP proxy over WebSocket in Go (Part 1)

Key takeaways

  • A reverse HTTP proxy over WebSocket is a type of proxy server which uses the WebSocket protocol as a "tunnel" to pass TCP communication from server to client.
  • In Go project, gorilla/websocket is widely used to implement WebSocket.
  • WebSocket is designed to work over HTTP. To achieve compatibility with HTTP, the WebSocket handshake uses the HTTP Upgrade header in order to change from the HTTP protocol to the WebSocket protocol.
  • Authentication of WebSocket is a difficult design problem, and there are many options.

Reverse HTTP proxy over WebSocket

There are two types of proxies, forward proxy and reverse proxy.

A forward proxy (or gateway, or tunnel, or just "proxy") provides proxy services to a client or a group of clients. It allow users to hide their IP address while accessing a Web service or an API server. In contract, a reverse proxy is the opposite of what a forward proxy does. It is a type of proxy server that retrieves resources on behalf of a client from servers.

Further, this post focuses on a reverse HTTP proxy over WebSocket, in brief, it uses the WebSocket protocol as a "tunnel" to pass TCP communication from server to client.

A network diagram for reverse proxy over WebSocket

A case study in Go

There are not that many examples of implementation reverse proxy over WebSocket. In JavaScript, mhzed/wstunnel is well known, in Haskell, erebe/wstunnel is.

In Go, inconshreveable/ngrok and coyove/goflyway is well known, especially ngrok is popular among developers as a SaaS service.

In this post, we will focus on understanding the basic concepts and read prototypical and simpler one root-gg/wsp. wsp is developed by root-gg, which is a reverse HTTP proxy over WebSocket, whose aim is to securely make call to internal APIs from outside. It is difficult to use in production, but it is a good learning material to explain the design of reverse proxy over WebSocket.

However, maintenance has stopped since the days of Go 1.6, so I'll proceed with this post based on code hgsgtk/wsp that I forked and modified for the Go situation in 2021 (Thank you root-gg).

GitHub logo hgsgtk / wsp

HTTP tunnel over Websocket

Demonstration of wsp

It's easier to get a general idea of how it works if you see it in action, so here's a brief demo. In the demonstration, the following four system component are introduced.

  1. Internal API(http://localhost:8081): the server you want to request via the reverse proxy
  2. wsp client: running in a network with the internal API
  3. wsp server(http://localhost:8080): running outside the network
  4. App: starting point for communication to the internal API

A diagram of system components quoted from README in root-gg/wsp

Try to send a request to the internal API via WebSocket communication to the endpoint /hello.

$ curl -H 'X-PROXY-DESTINATION: http://localhost:8081/hello' http://127.0.0.1:8080/request
Enter fullscreen mode Exit fullscreen mode

HTTP communication was relayed by the following route.

app -> wsp server -(WebSocket)-> wsp client -> internal API
Enter fullscreen mode Exit fullscreen mode

Here is a terminal image when sending a HTTP request from app.

A terminal image when sending a HTTP request from app

A terminal image when wsp client got a message from wsp server

The internal network restricts incoming requests from outside or does not have any global IPs, so that it is not possible for external server to connect to a client started in an internal network. For example, if the client is launched on your local PC, it is not possible to send a request to the client from an external server.

A diagram describing the internal network restricts requests from outside

Therefore, the starting point is wsp client in the internal network. In this design, it starts with a request for a WebSocket connection from wsp client to wsp server.

A diagram describing to establish WebSocket connection from client

The following points to implement this design will be explained in this post.

  • Start a WebSocket server

I will explain the rest points in part 2 and beyond.

  • Establish a WebSocket connection
  • Keep a established connection
  • Relay TCP connection from "App" to the peer of WebSocket
  • Relay TCP connection in WebSocket data to "internal API"

Start a WebSocket server

The first step is to start up a WebSocket server which waits for requests from WebSocket clients.

As a side note, WebSocket is a mechanism for low-cost, full-duplex communication on Web, which protocol was standardized as RFC 6455. The following diagram, quoted by Wikipedia, describe a communication using WebSocket between client and server.

A diagram describing a connection using WebSocket

In the case of wsp, it starts like this.

$ ./wsp_server -config examples/wsp_server.cfg
{
  "Host": "127.0.0.1",
  "Port": 8080,
  "Timeout": 1000,
  "IdleTimeout": 60000,
  "Whitelist": [],
  "Blacklist": [],
  "SecretKey": ""
}
Enter fullscreen mode Exit fullscreen mode

Let's read the Go code. First is the main function (cmd/wsp_server/main.go), which is the entry point.

package main

import (
    // (omit)
)

func main() {
    // (omit)

    server := server.NewServer(config)
    server.Start()

    // (omit: shutdown)
}
Enter fullscreen mode Exit fullscreen mode

server.NewServer function returns a pointer of Server struct which represents a reverse HTTP proxy over WebSocket.

type Server struct {
    Config *Config

    upgrader websocket.Upgrader

    pools []*Pool
    lock  sync.RWMutex
    done  chan struct{}

    dispatcher chan *ConnectionRequest

    server *http.Server
}
Enter fullscreen mode Exit fullscreen mode

An important field is upgrader whose type is websocket.Upgrader to specify parameters for upgrading an HTTP connection to a WebSocket connection. In Go project, gorilla/websocket is widely used to implement WebSocket, and many samples using this library are available on the internet.

WebSocket is designed to work over HTTP. To achieve compatibility with HTTP, the WebSocket handshake uses the HTTP Upgrade header in order to change from the HTTP protocol to the WebSocket protocol.

Fields of Upgrader struct is as follow:

type Upgrader struct {
    HandshakeTimeout time.Duration
    ReadBufferSize, WriteBufferSize int
    Subprotocols []string
    Error func(w http.ResponseWriter, r *http.Request, status int, reason error)
    CheckOrigin func(r *http.Request) bool
    EnableCompression bool
}
Enter fullscreen mode Exit fullscreen mode

If you want to know more about the WebSocket parameters or gorilla/websocket internal implementation, refer to this post Learn WebSocket handshake protocol with gorilla/websocket server.

In NewServer function, it creates a value whose type is websocket.Upgrader with default parameters.

func NewServer(config *Config) (server *Server) {
    // (omit)

    server.upgrader = websocket.Upgrader{}

    // (omit)
    return
}
Enter fullscreen mode Exit fullscreen mode

Next, call server.Start function to start a WebSocket server.

func (server *Server) Start() {
    // (omit)

    r := http.NewServeMux()
    r.HandleFunc("/request", server.request)

    // (omit)

    server.server = &http.Server{
        Addr:    server.Config.GetAddr(),
        Handler: r,
    }
    go func() { log.Fatal(server.server.ListenAndServe()) }()
Enter fullscreen mode Exit fullscreen mode

In this code, an HTTP server that exposes an endpoint /request is set up at a given address. The handler mapped to /request is as follow (code):

func (server *Server) register(w http.ResponseWriter, r *http.Request) {
    secretKey := r.Header.Get("X-SECRET-KEY")
    if secretKey != server.Config.SecretKey {
        wsp.ProxyErrorf(w, "Invalid X-SECRET-KEY")
        return
    }

    ws, err := server.upgrader.Upgrade(w, r, nil)
    if err != nil {
        wsp.ProxyErrorf(w, "HTTP upgrade error : %v", err)
        return
    }

    // (omit)
}
Enter fullscreen mode Exit fullscreen mode

There are several knowledges to implement WebSocket server, so I'll explain it line by line.

Authentication of WebSocket

The first line is:

secretKey := r.Header.Get("X-SECRET-KEY")
if secretKey != server.Config.SecretKey {
    wsp.ProxyErrorf(w, "Invalid X-SECRET-KEY")
    return
}
Enter fullscreen mode Exit fullscreen mode

The WebSocket protocol does not handle authorization or authentication. However, it is a problem that anyone can access the WebSocket server without authentication just because the protocol is not prepared.

Business requirements often require authenticating the connecting WebSocket client and associating it with a registered customer record.

There are several patterns to implement authentication, and Heroku Dev Center - WebSocket Security describes one pattern - "ticket" -based authentication system.

  1. The client-side contacts the server-side to obtain an authorization "ticket".
  2. The server stores the ticket and returns it to the client
  3. The client opens the WebSocket connection, and sends along this "ticket" as part of an initial handshake.
  4. The server can compare this ticket, check an user's condition (i.e. source IPs)

See Authorization and IPs chapter in Armin Ronacher's post Websockets 101 for detail.

There are several options for where and how to tell the "ticket" from the client to the server. A commonly used method for simple authentication is to set the relevant ticket in the HTTP header. For example, Amazon API Gateway provides the option to require API clients to pass the API key as the X-API-Key header.

The sample code implements a similar concept. It validates the value in the header X-SECRET-KEY at initial handshake.

secretKey := r.Header.Get("X-SECRET-KEY")
Enter fullscreen mode Exit fullscreen mode

Also, depending on your business requirements, there are several options for how to issue the "ticket". A idea is to issue "ticket" (or access key, or access token) in advance on their administration UI. Or if the process of asking where the WebSocket server to connect contains before handshake, it is also good idea to issue a one-time "ticket" by "access token" which is issued in advance.

In "ticket" -based authentication system, the server authenticates only at initial handshake, but there is also an idea to authenticate every message. See Klee Thomas's post Authenticating Websockets more detail.

Handshake (HTTP Upgrade)

After authentication, upgrade an HTTP connection to an WebSocket connection.

ws, err := server.upgrader.Upgrade(w, r, nil)
if err != nil {
    wsp.ProxyErrorf(w, "HTTP upgrade error : %v", err)
    return
}
Enter fullscreen mode Exit fullscreen mode

websocket.Upgrader.Upgrade function does it. To put it simply, it checks if the handshake request from the client is valid and returns a successful handshake response if it is.

i.e. a valid handshake request

GET /register HTTP/1.1
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==
Origin: http://example.com
Sec-WebSocket-Version: 13
Enter fullscreen mode Exit fullscreen mode

i.e. a successful handshake response

HTTP/1.1 101 Switching Protocols
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Accept: s3pPLMBiTxaQ9kYGzzhZRbK+xOo=
Enter fullscreen mode Exit fullscreen mode

If you want to know the internal implementation of Upgrader.Upgrade function, see Learn WebSocket handshake protocol with gorilla/websocket server.

Conclusion

This post explained how to start a WebSocket server and WebSocket specification and security considerations to establish a WebSocket connection.

I will explain the rest points in part 2 and beyond.

  • Establish a WebSocket connection (Part 2)
  • Keep a established connection
  • Relay TCP connection from "App" to the peer of WebSocket
  • Relay TCP connection in WebSocket data to "internal API"

Discussion (3)

Collapse
b0r profile image
b0r

Hi Kazuki,

this is a really nice post about implementing reverse proxy!! :thumbs-up:

May I just ask you to increase the size of the "Here is a terminal image when sending a HTTP request from app." to make it more readable, please?

Collapse
hgsgtk profile image
Kazuki Higashiguchi Author • Edited on

I got it! Thanks for reading it!
I’m going to fix the image tomorrow✍️

Collapse
hgsgtk profile image
Kazuki Higashiguchi Author

I's done!