Test Driven Development for Docker Images

Containers were one of the driving forces behind the modern DevOps culture. Where there used to be a strict barrier between developers and operations, is now a common, agreed upon standard. Developers are able to configure their runtime environment to their needs, focusing on what happens inside a container, IT-Ops takes care of things outside a container - how to run, scale and monitor them.
While the concept of process isolation has been around for much longer, the one thing which brought container technology to the masses was Docker in 2013. Gone should be the days when developers handed over some kind of deliverable and IT-Ops had to somehow take care of it. No more “but it works on my machine” - if it works on your machine, we will ship your machine!

With great power comes great responsibility

Given the ability to customise the runtime environment, responsibility for providing and maintaining a proper environment shifts from IT-Ops towards developers. We’re not only responsible for our application, but also the environment to run it in. Fortunate enough, we can apply well-known methods to make this effort less tedious: automated tests and test-driven development (TDD).

Meet Goss and DGoss

Goss is a neat little utility which helps you validate your server configuration.
As stated on its GitHub page it’s a YAML based alternative to Ruby-based serverspec. (And since it’s written in Go, I assume Goss is short for Go serverspec)

So, how will Goss help us test our Docker images? With Goss itself it's possible to verify a multitude of properties on Linux systems, ranging from filesystem layouts, permissions and content over network ports, installed packages for various distributions to running services. This would allow us to e.g. verify the content of our Apache config, that our apache2 service is actually enabled and running and last but not least, whether it’s listening on port 80.

Now when building a Docker image, we can apply these things too. DGoss is a wrapper script around Goss which allows us to target Docker containers, so let’s explore how we can write Docker tests.

Installation

As mentioned earlier, Goss and DGoss currently only support Linux systems. Since we want to target Linux containers, this is not a problem for us. We can get the latest release on GitHub:

curl -L https://github.com/aelsabbahy/goss/releases/latest/download/goss-linux-amd64 -o /usr/local/bin/goss
chmod +rx /usr/local/bin/goss

curl -L https://github.com/aelsabbahy/goss/releases/latest/download/dgoss -o /usr/local/bin/dgoss
chmod +rx /usr/local/bin/dgoss

In case you should decide to download Goss to a location outside your PATH, you’ll have to provide an additional environment variable GOSS_PATH, which points to the executable.

First Glance

To get a first impression, we will create a short test for the httpd Docker image. If not specified otherwise via GOSS_FILE environment variable, the default spec file is called goss.yaml:

port:
  tcp:80:
    listening: true
    ip:
    - 0.0.0.0
http:
  http://localhost:
    status: 200
    body: ["It works!"]

This exemplary test verifies that we have a server running on TCP port 80, listening on all IP addresses. Additionally, it makes sure it serves the default page which only displays ”It works!”.

To run our tests we execute dgoss run httpd instead of docker run httpd. Dgoss will start a container based on our specified image and copy all required files for us. Assuming the image under test works like expected we should see them succeed:

INFO: Starting docker container
INFO: Container ID: 76715e17
INFO: Sleeping for 0.2
INFO: Container health
INFO: Running Tests
Port: tcp:80: listening: matches expectation: [true]
Port: tcp:80: ip: matches expectation: [["0.0.0.0"]]
HTTP: http://localhost: status: matches expectation: [200]
HTTP: http://localhost: Body: matches expectation: [It works!]


Total Duration: 0.010s
Count: 4, Failed: 0, Skipped: 0
INFO: Deleting container

Test Driven Docker Development (TDDD)

Now the above example is not that interesting, let’s instead focus on building an image for e.g. a node backend. We will be using a Debian base image, debian:10.4-slim to be precise, and change the default shell to bash for our build.

FROM debian:10.4-slim

SHELL [ "/bin/bash", "-c" ]

Next to our Dockerfile we create a goss.yaml file as well as a little helper script to build and test our image, build_and_test.sh.

#!/usr/bin/env bash

set -e

IMAGE=s1hofmann/node-runtime
TAG=12.18.0

docker build -t $IMAGE:$TAG .
dgoss run -it $IMAGE:$TAG

The repository linked at the end of the post also contains a nodemon setup to automatically build and test the image on file changes.

Non-root images

The first thing we want to add to our image is a dedicated non-root user and group to run our app, so let’s add both a group and user test to our Goss file.

group:
  node:
    exists: true
    gid: 1000
    skip: false

user:
  node:
    exists: true
    uid: 1000
    gid: 1000
    groups:
    - node
    home: /home/node
    shell: /bin/bash
    skip: false

Since we did not yet apply any changes to our base image, our tests will fail:

+ IMAGE=s1hofmann/node-runtime
+ TAG=12.18.0
+ docker build -t s1hofmann/node-runtime:12.18.0 .
Sending build context to Docker daemon  4.096kB
Step 1/2 : FROM debian:10.4-slim
 ---> 108d75da320f
Step 2/2 : SHELL [ "/bin/bash", "-c" ]
 ---> Using cache
 ---> 39a73e053958
Successfully built 39a73e053958
Successfully tagged s1hofmann/node-runtime:12.18.0
+ dgoss run -it s1hofmann/node-runtime:12.18.0
INFO: Starting docker container
INFO: Container ID: ae8fbadc
INFO: Sleeping for 0.2
INFO: Container health
INFO: Running Tests
Group: node: exists:
Expected
    <bool>: false
to equal
    <bool>: true
Group: node: gid: skipped
User: node: exists:
Expected
    <bool>: false
to equal
    <bool>: true
User: node: uid: skipped
User: node: gid: skipped
User: node: home: skipped
User: node: groups: skipped
User: node: shell: skipped


Failures/Skipped:

Group: node: exists:
Expected
    <bool>: false
to equal
    <bool>: true
Group: node: gid: skipped

User: node: exists:
Expected
    <bool>: false
to equal
    <bool>: true
User: node: uid: skipped
User: node: gid: skipped
User: node: home: skipped
User: node: groups: skipped
User: node: shell: skipped

Total Duration: 0.003s
Count: 8, Failed: 2, Skipped: 6
INFO: Deleting container

Once we extended our image, our tests should pass:

FROM debian:10.4-slim

SHELL [ "/bin/bash", "-c" ]

RUN groupadd --gid 1000 node && useradd --uid 1000 --gid node --shell /bin/bash --create-home node

+ IMAGE=s1hofmann/node-runtime
+ TAG=12.18.0
+ docker build -t s1hofmann/node-runtime:12.18.0 .
Sending build context to Docker daemon  4.096kB
Step 1/3 : FROM debian:10.4-slim
 ---> 108d75da320f
Step 2/3 : SHELL [ "/bin/bash", "-c" ]
 ---> Using cache
 ---> 39a73e053958
Step 3/3 : RUN groupadd --gid 1000 node && useradd --uid 1000 --gid node --shell /bin/bash --create-home node
 ---> Using cache
 ---> 8700babb2f3b
Successfully built 8700babb2f3b
Successfully tagged s1hofmann/node-runtime:12.18.0
+ dgoss run -it s1hofmann/node-runtime:12.18.0
INFO: Starting docker container
INFO: Container ID: 4f21c178
INFO: Sleeping for 0.2
INFO: Container health
INFO: Running Tests
Group: node: exists: matches expectation: [true]
Group: node: gid: matches expectation: [1000]
User: node: exists: matches expectation: [true]
User: node: uid: matches expectation: [1000]
User: node: gid: matches expectation: [1000]
User: node: home: matches expectation: ["/home/node"]
User: node: groups: matches expectation: [["node"]]
User: node: shell: matches expectation: ["/bin/bash"]


Total Duration: 0.003s
Count: 8, Failed: 0, Skipped: 0
INFO: Deleting container

With our first tests passing we still want to make sure our image actually switches to the created user. A command test can be used to verify this:

command:
  container_user:
    exec: "id -un" 
    exit-status: 0
    stdout:
    - node
    skip: false
  container_group:
    exec: "id -gn" 
    exit-status: 0
    stdout:
    - node
    skip: false
  container_pwd:
    exec: "pwd" 
    exit-status: 0
    stdout:
    - /home/node
    skip: false

As expected, our new tests will initially fail, but once we extend our Dockerfile, they should pass again:

FROM debian:10.4-slim

RUN groupadd --gid 1000 node && useradd --uid 1000 --gid node --shell /bin/bash --create-home node

USER node:node
WORKDIR /home/node

Count: 14, Failed: 0, Skipped: 0

Installed Packages

Our upcoming node setup requires either of wget or curl. It is certainly not required to keep them in our image, but since curl has been really useful more than once, I like to have it pre-installed. Let’s add a package test for it:

package:
  curl:
    installed: true
    skip: false

To fix our now failing test, let’s also update our Dockerfile:

FROM debian:10.4-slim

SHELL [ "/bin/bash", "-c" ]

RUN groupadd --gid 1000 node && useradd --uid 1000 --gid node --shell /bin/bash --create-home node

RUN apt-get update \
    && apt-get install -y curl \
    && rm -rf /var/lib/apt/lists/*

USER node:node
WORKDIR /home/node

node

nvm turned into my go-to solution to install node. When installing node using nvm, we want to make sure that

1. A .nvm folder is located in our home directory
2. A node version corresponding to our image tag is installed at $HOME/.nvm/versions/node
3. Our node version is set as default in $HOME/.nvm/alias/default

A possible way to verify this is a file test which uses variables:

file:
  /{{ .Env.HOME }}/.nvm:
    exists: true
    owner: node
    group: node
    filetype: directory
  /{{ .Env.HOME }}/.nvm/versions/node/v{{ .Env.NODE_VERSION }}:
    exists: true
    owner: node
    group: node
    filetype: directory
  /{{ .Env.HOME }}/.nvm/alias/default:
    exists: true
    owner: node
    group: node
    filetype: file
    contains: [/^{{ .Env.NODE_VERSION }}$/]

The following additions to our Dockerfile are required to pass our latest test:

ARG NODE_VERSION=12.18.0
ENV NODE_VERSION $NODE_VERSION
RUN curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.35.3/install.sh | bash && chmod +x $HOME/.nvm/nvm.sh && . $HOME/.nvm/nvm.sh && nvm install $NODE_VERSION && nvm alias default $NODE_VERSION && nvm use default

Startup

Our image should provide an entrypoint script which either forwards all non-command parameters to node, or executes any other command so we’re able to start our container running e.g. a bash shell.
To make sure we're providing an entrypoint script we’re going to extend our file test:

file:
  ...
  /usr/local/bin/entrypoint.sh:
    exists: true
    owner: node
    group: node
    filetype: file

And with our entrypoint in place, we expect a running node process in our container:

process:
  node:
    running: true

In order to pass our last tests we have will have to copy our script and configure an entrypoint:

COPY --chown=node:node entrypoint.sh /usr/local/bin

ENTRYPOINT [ "entrypoint.sh" ]
CMD [ "node" ]

Conclusion

In this post we built a node Docker image the test-driven way. Instead of manually validating our setup at runtime we were able to write tests for our setup upfront, which made it easier for us to verify our image configuration.

The resulting image is configurable, so we’re free to choose our runtime, but still comes with a safety net which makes sure everything is in place. Best practices known from software development also apply to building Docker images!

The whole setup can be found on GitHub, if you have any questions, feel free to get in touch with me!