Using Docker Compose Entrypoint To Check if Postgres is Running

Getting to know Docker

Over the last several weeks I have been working to get to know Docker at a much deeper level. Even after using it off and on for a year or so now, I still feel like I have much to learn. It is an ever changing landscape, but it's really a lot of fun to learn at a deeper level.

I had been struggling with understanding how to start up and application, and ensure that Postgres was running. I was experiencing this issue while working on some new tutorials for Codeship. Let's walk through the problem together, and how it manifests itself.

The Problem

Docker has a couple methods to boot your containers in order, one of those being depends_on. Typically - this is a great solution, but it also doesn't really tell you if the service is running, just that the container has started.

Docker explains it this way:

The problem of waiting for a database (for example) to be ready is really just a subset of a much larger problem of distributed systems. In production, your database could become unavailable or move hosts at any time. Your application needs to be resilient to these types of failures.

My Dockerfile was fairly basic as well -

FROM node:7.7.2-alpine

COPY package.json /tmp/package.json
RUN cd /tmp && npm install --quiet
RUN mkdir -p /usr/app && cp -a /tmp/node_modules /usr/app

WORKDIR /usr/app
COPY ./ /usr/app/

Nothing too crazy - just installing npm modules, copying the app files over, and then I let compose run the commands to start the app itself. In order to start my app, Postgres had to run a quick check to verify the table was set up and ready to roll.

Where I was running into trouble is that I would see occasional failures with the pg module not able to connect to the client, typically when rebuilding the images, and during testing specifically. What I would notice is that the Postgres container was still getting itself ready to take connections, and the node app was already trying to start testing - leading to a race condition.

It's important to note, that using Codeship to deploy the app to Heroku, depends_on isn't an option for me, so I'm using links to connect everything. Since we aren't really guaranteed anything with depends_on either, I'm ok with that, and it hasn't contributed to the issue, but the tests must work or I'll never get a deployment to run.

The Solution

Let me preface this with a couple things:

1. Docker can solve this multiple ways. I don't know them all, but this is a viable solution.
2. I am always looking for new methods, so comments that are constructive are welcomed.
3. Docker is always changing, so this may be out of date if you are reading this further out from the original post date.

Entrypoint

I don't plan on getting exhaustive with the entrypoint review here, however what I've been able to learn so far is that in combination with CMD, it will essentially work as a wrapper script that can run a script after something has been fulfilled.

The idea here is that I can wait for psql to respond, and then I can start my app. I'm still working through some of the details but the script looks like this:

#!/bin/sh
# wait-for-postgres.sh

set -e

cmd="$@"

while ! pg_isready -h "postgres" -p "5432" > /dev/null 2> /dev/null; do
   echo "Connecting to postgres Failed"
   sleep 1
 done

>&2 echo "Postgres is up - executing command"
exec $cmd

Between documentation, and Stack Overflow, I was able to piece together a shell script to use the pg_isready command to determine if the postgres container is accepting connections or not. Once it is, then I can pass any command that will execute, and become PID 1 because I'm using exec $cmd to run npm test.

Modifications

I did need to modify some things. I was originally using the alpine image which is using Busy Box, a super lightweight Linux flavor. The difficulty I had was that I needed to install the postgres tooling to have the pg_isready function available. I switched to the slim image, and modified my Dockerfile a bit.

FROM node:7.7.2-slim

RUN apt-get update && apt-get install -f -y postgresql-client
COPY package.json /tmp/package.json
RUN cd /tmp && npm install --quiet
RUN mkdir -p /usr/app && cp -a /tmp/node_modules /usr/app

WORKDIR /usr/app
COPY ./ /usr/app/
ENTRYPOINT ["./wait-for-postgres.sh"]

Three main changes:

1. Changing to the node:7.7.2-slim image to have more Linux tools available.
2. RUN apt-get update && apt-get install -f -y postgresql-client is run to install the tooling needed for the wait-for-postgres.sh script to work.
3. ENTRYPOINT ["./wait-for-postgres.sh"] to run the shell script, and then whatever command I pass to it is run once the loop exits in the script.

Now - I have the ability to run npm run dev or npm test and know that postgres is ready to go.

Wrap up

I'm still working on things - but wanted to share this first. E.g. - test that the tables are also setup before executing the app, explore better retry logic in the node app itself, and probably more - but this works.

It's exciting to learn about everything Docker and how to deliver on Codeship. Like everything, there are ups and downs, so to speak. At times the debugging can be tedious, but it's a small price to pay for learning this.

Looking forward to your comments - here or @kellyjandrews on Twitter.