Build a 2MB REST API -- Scratch Images Episode III

#docker #tutorial #go

The last episode, 100kB Docker Images with Static ELF binaries was a hit. Before proceeding, check that out.

Here we'll do something more illustrative and useful by building a simple Go REST API proxy -- packed into a tiny 2MB. This will show that we can do more than just hello world, and illustrate how to add other dependencies into the scratch image.

The REST API

This proxy fetches the resource at the url and returns it. This is a common pattern for caching remote resources.

e.g.:

$ curl  'http://localhost:8080/?url=https%3A%2F%2Fwww.httpbin.org%2Fget' \
| jq .headers
{
  "Accept-Encoding": "gzip",
  "Host": "www.httpbin.org",
  "User-Agent": "Go-http-client/1.1"
}

main.go

package main

import (
    "fmt"
    "io"
    "log"
    "net/http"
    "os"
)

func main() {
    http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
        url := r.FormValue("url")
        if url == "" {
            w.WriteHeader(http.StatusBadRequest)
            w.Write([]byte("\"url\" param is required"))
            return
        }
        resp, err := http.Get(url)
        if err != nil {
            w.WriteHeader(http.StatusInternalServerError)
            w.Write([]byte(fmt.Sprintf("Error fetching url [%s]: %s", url, err.Error())))
            return
        }
        // stream the resp body to our HTTP response, w
        writtenCount, err := io.Copy(w, resp.Body)
        if err != nil || writtenCount == 0 {
            w.WriteHeader(http.StatusInternalServerError)
            w.Write([]byte("Response was empty from url  " + url))
            return
        }
    })
    if port := os.Getenv("PORT"); port != "" {
        http.ListenAndServe(":"+port, nil)
    } else {
        log.Panic("PORT not set")
    }
}

At a high level, we implement a handler passed to HandleFunc which calls http.Get(url) on the url query param, and then stream the urls body to our client.

Finally we listen to the PORT passed by environment.

The Dockerfile

FROM golang:stretch AS build
WORKDIR /build
RUN apt-get update && \
    apt-get install -y xz-utils
ADD https://github.com/upx/upx/releases/download/v3.95/upx-3.95-amd64_linux.tar.xz /usr/local
RUN xz -d -c /usr/local/upx-3.95-amd64_linux.tar.xz | \
    tar -xOf - upx-3.95-amd64_linux/upx > /bin/upx && \
    chmod a+x /bin/upx
COPY . .
RUN GO111MODULE=off CGO_ENABLED=0 GOOS=linux \
    go build  -a -tags netgo -ldflags '-w -s'  main.go && \
    upx main

FROM scratch
COPY --from=build /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/
COPY --from=build /build/main /main
WORKDIR /
CMD ["/main"]

Here we show a few new concepts:

For go, static builds require -tags netgo . If your build ends up with other dynamic libs, you can use CGO_ENABLED=1 ... -extldflags "-static" -- which requires the C toolchain.
upx -- The Ultimate Packer for Exes is used to reduce about 70%
As we are making https requests (during the http.Get() call), we will need our client certificate bundle, ca-certificates.crt which allows us to authenticate the https server's cert.

Build the Image

docker build . -t go-http-proxy

docker images |grep go-http-proxy | awk '{print $7}'   
2.16MB

A bit more than 100kb, but 2MB ain't bad! It's 99.5% smaller than the 806MB stretch image we would have been using.

docker images |grep c0167164f9fa | awk '{print $7}' 
806MB

Running and Testing

run...

docker run -ePORT=8080 -p8080:8080 go-http-proxy

test...

curl  'http://localhost:8080/?url=https%3A%2F%2Fwww.httpbin.org%2Fget' | jq .headers
{
  "Accept-Encoding": "gzip",
  "Host": "www.httpbin.org",
  "User-Agent": "Go-http-client/1.1"
}

What's Inside

This image has two layers (one each per COPY) -- let's take a look

$ mkdir go-http-proxy &&  cd go-http-proxy 
$ docker save go-http-proxy |tar -x
$ find . -iname layer.tar|xargs -n 1 tar -tf
main
etc/
etc/ssl/
etc/ssl/certs/
etc/ssl/certs/ca-certificates.crt

... still just the two files.

Next Steps

At this point we'll need to level up our game to scripting languages like nodejs -- which contain far more dependencies both within your app and in the runtime.

➡️ Let's hear your tips for shrinking those docker images. And what platforms would you like to see built from scratch?