Guide: Rails Development with Docker

Originally posted on Hint's blog.

As a software consultancy, we switch between many projects throughout the year. A critical factor in delivering value is the ease at which we are able to move between projects.

Over the years, we have used many tools to manage dependencies needed to run and develop our clients' projects. The problem with most tools has been the ability to have consistent, reproducible development environments across our team. About two years ago, we discovered that Docker was a viable option for building consistent development environments. Since then, we continue to iterate on our configuration as we learn new ways to handle the complexity of the projects while simplifying the setup process for our team.

In this guide, we will cover the basics of our Docker development environment for Rails.

Getting started

If you would like to follow along, install Docker CE and create, clone, or have a working Rails app.

We will be using a combination of Dockerfiles, Docker Compose, and bash scripts throughout this guide, so let's make a place for most of those files to live.
Start by creating a docker folder in the root of the Rails project. Here we will store Dockerfiles and bash scripts to be referenced from the docker-compose.yml file we will be creating later.

Inside the docker folder, create another folder named ruby. Inside the newly created ruby folder, create a file named Dockerfile. This file will contain commands to build a custom image for your Rails app.

Hello Dockerfile

  ARG RUBY_VERSION=2.6
  FROM ruby:$RUBY_VERSION
  ARG DEBIAN_FRONTEND=noninteractive

In this block, we set the RUBY_VERSION and DEBIAN_FRONTEND build arguments and specify the docker image we will use as our base image.

The first ARG sets a default value for RUBY_VERSION, but passing in a value from the command line or a docker-compose file will override it, as you'll see later in the guide. An ARG defined before FROM is only available for use in FROM.

FROM in a Dockerfile is the base image for building our image and the start of the build stage. In our case, we are using the official Ruby image from Docker Hub, defaulting to Ruby 2.6.

The next ARG is used to set the build stage shell to noninteractive mode, which is the general expectation during the build process. We don't want this environment variable to carry over to when we are using the images in our development environment, which is why we are using ARG instead of ENV.

Base Software Install

  ARG NODE_VERSION=11
  RUN curl -sL https://deb.nodesource.com/setup_$NODE_VERSION.x | bash -
  RUN curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | apt-key add - && \
    echo "deb https://dl.yarnpkg.com/debian/ stable main" | tee /etc/apt/sources.list.d/yarn.list

  RUN apt-get update && apt-get install -y \
    build-essential \
    nodejs \
    yarn \
    locales \
    git \
    netcat \
    vim \
    sudo

Here we are setting up the necessary software and tools for running a modern Rails app. Like in the previous section, we are setting a default for our NODE_VERSION build argument.

The next two RUN lines set up the defined version of the node apt repository and the latest stable yarn apt repo. Since Webpacker started being officially supported in Rails 5.1, it is important we have a recent version of node and yarn available in the image we will be running Rails on.

The third RUN will look familiar if you have used any Debian based OS. It updates the apt repositories, which is important since we just installed two new ones, and then it installs our base software and tools.

I want to point out netcat and sudo specifically. netcat is a networking tool we will use to verify the other services are up when we are bringing up our Rails app through Docker Compose. We install sudo since by default it is not installed on the Debian based Docker images, and we will be using a non-root user in our Docker image.

Non-root User

  ARG UID
  ENV UID $UID
  ARG GID
  ENV GID $GID
  ARG USER=ruby
  ENV USER $USER

  RUN groupadd -g $GID $USER && \
      useradd -u $UID -g $USER -m $USER && \
      usermod -p "*" $USER && \
      usermod -aG sudo $USER && \
      echo "$USER ALL=NOPASSWD: ALL" >> /etc/sudoers.d/50-$USER

Docker containers generally use the root user, which is not inherently bad, but is problematic for file permissions in a development environment. Our solution for this is to create a non-root user and pass in our UID, GID, and username as build arguments.

If we pass in our UID and GID, all files created or modified by the user in the container will share the same permissions as our user on the host machine.

You will notice here that we use the build arguments to set the environment variable (via ENV) since we will also want these variables available when we bring up the container.

In the RUN instruction we add a standard Linux user/group and then we add the new user to the sudoers file with no password. This gives us all the benefits of running as root while keeping file permissions correct.

Ruby, RubyGems, and Bundler Defaults

  ENV LANG C.UTF-8

  ENV BUNDLE_PATH /gems
  ENV BUNDLE_HOME /gems

  ARG BUNDLE_JOBS=20
  ENV BUNDLE_JOBS $BUNDLE_JOBS
  ARG BUNDLE_RETRY=5
  ENV BUNDLE_RETRY $BUNDLE_RETRY

  ENV GEM_HOME /gems
  ENV GEM_PATH /gems

  ENV PATH /gems/bin:$PATH

Explicitly setting the LANG environment variable specifies the fallback locale setting for the image. UTF-8 is a sane fallback and is what locale defaults to when it's working properly.

We will be using a volume with Compose, so we need to point RubyGems and Bundler to where that volume will mount in the file system. We also set the gem executables in the path and set some defaults for Bundler, which are configurable via build arguments.

Optional Software Install

  #-----------------
  # Postgres Client:
  #-----------------
  ARG INSTALL_PG_CLIENT=false

  RUN if [ "$INSTALL_PG_CLIENT" = true ]; then \
      apt-get install -y postgresql-client \
  ;fi

Here is an example of how to set up optional software installs in the Dockerfile. In this scenario, postgresql-client will not be installed by default, but will be installed if we pass a build argument set to true (INSTALL_PG_CLIENT=true).

Dockerfile Final Touches

  RUN mkdir -p "$GEM_HOME" && chown $USER:$USER "$GEM_HOME"
  RUN mkdir -p /app && chown $USER:$USER /app

  WORKDIR /app

  RUN mkdir -p node_modules && chown $USER:$USER node_modules
  RUN mkdir -p public/packs && chown $USER:$USER public/packs
  RUN mkdir -p tmp/cache && chown $USER:$USER tmp/cache

  USER $USER

  RUN gem install bundler

To wrap up the Dockerfile, we create and set permissions on needed directories that were referenced previously in the file or that we will be setting up as volumes with Docker Compose.

We call USER $USER here at the end of the file, which will set the user for the image when you boot it as a container.

The last RUN command installs Bundler, which may not be required depending on the version of Ruby.

Next, we'll take a look at our docker-compose.yml file.

Docker Compose

Docker's definition of Compose is a great place to start.

Compose is a tool for defining and running multi-container Docker applications. With Compose, you use a YAML file to configure your application's services. Then, with a single command, you create and start all the services from your configuration.

That is what we are going to be doing in this section of the guide, adding support for a multi-container Docker application.

The first thing we will need is a docker-compose.yml file at the root of the Rails app. If you would like to copy and paste the following YAML into a docker-compose.yml file, I'll breakdown its parts below.

  version: '3.7'

  services:
    rails:
      build:
        context: ./docker/ruby
        args:
          - RUBY_VERSION=2.6
          - BUNDLE_JOBS=15
          - BUNDLE_RETRY=2
          - NODE_VERSION=12
          - INSTALL_PG_CLIENT=true
          - UID=500
          - GID=500
      environment:
        - DATABASE_USER=postgres
        - DATABASE_HOST=postgres
      command: bundle exec rails server -p 3000 -b '0.0.0.0'
      entrypoint: docker/ruby/entrypoint.sh
      volumes:
        - .:/app:cached
        - gems:/gems
        - node_modules:/app/node_modules
        - packs:/app/public/packs
        - rails_cache:/app/tmp/cache
      ports: 
        - "3000:3000"
      user: ruby
      tty: true
      stdin_open: true
      depends_on:
        - postgres
    postgres:
      image: postgres:11
      environment:
        - POSTGRES_HOST_AUTH_METHOD=trust
      volumes:
        - postgres:/var/lib/postgresql/data

  volumes:
    gems:
    postgres:
    node_modules:
    packs:
    rails_cache:

In this file, we are defining two services: the rails service, which our Rails app will run in, and the postgres service, which will accommodate PostgreSQL. The names for these services are arbitrary and could easily be foo and bar, but since the service names are used for building, starting, stopping, and networking, I would recommend naming them close to the actual service they are running.

We also are setting our Compose file compatibility to version: '3.7' which is the latest at the time of this writing.

The App Service Broken Down

  version: '3.7'

  services:
    rails:
      build:
        context: ./docker/ruby
        args:
          - RUBY_VERSION=2.6
          - BUNDLE_JOBS=15
          - BUNDLE_RETRY=2
          - NODE_VERSION=12
          - INSTALL_PG_CLIENT=true
          - UID=500
          - GID=500

The first part of the rails service is to set up to build the image from the Dockerfile we created earlier. Once built, it uses the local image from that point forward. We set the context, which is the directory path where our Dockerfile is stored. The args: key specifies the build arguments we set up in the Dockerfile earlier in the guide. Notice that we are overriding some of our earlier defaults here.

  services:
    rails:
      . . .
      environment:
        - DATABASE_USER=postgres
        - DATABASE_HOST=postgres
      command: bundle exec rails server -p 3000 -b '0.0.0.0'
      entrypoint: docker/ruby/entrypoint.sh
      volumes:
        - .:/app:cached
        - gems:/gems
        - node_modules:/app/node_modules
        - packs:/app/public/packs
        - rails_cache:/app/tmp/cache
      ports: 
        - "3000:3000"

In the next part, we set up environment variables for connecting to the postgres service, which will require updating your database.ymlexample file to take advantage of these variables.

We also set the default command that will run when docker-compose up runs, which in this case starts the Rails server on port 3000 and binds it to all IP addresses.

Next, we point to our entrypoint script, which we'll revisit after breaking down the rest of the Compose file.

After the entrypoint, we set up volumes.

The first volume mounts the current directory (.) to /app in the container. The mapping for this one is HOST:CONTAINER. If you are on Mac, I would recommend using .:/app:cached since file sharing on Docker for Mac is CPU bound. By setting :cached on the mount point it allows files to be out of sync with the host being authoritative. Here are more details about file sharing performance on Docker for Mac. In our testing :cached has been the best option for speed vs. tradeoffs.

The next volumes are named volumes. Creating them happens when we bring up the containers for the first time and then they are persistent from up and down. These volumes are native to the Docker environment, so they operate at native speeds. The persistence and speed are why we have chosen to use them for gem, node_modules, packs, and rails_cache storage; otherwise, we would have to reinstall both every time we bring our environment back up. Lastly, there is a top-level volumes key at the very bottom of the docker-compose.yml file, which is where they are defined.

The last key in this section is ports. Ports only need to be defined to access the containers from the host otherwise container to container communication happens on the internal Docker network generally using service names.

The mapping "3000:3000" allows us to connect to the Rails server at [localhost:3000](http://localhost:3000). The mapping is HOST:CONTAINER just like the volume mount, and it is recommended to pass them in as strings because YAML parses numbers in xx:yy as base-60.

  services:
    rails:
      . . .
      user: ruby
      tty: true
      stdin_open: true
      depends_on:
        - postgres

This last section explicitly sets the user to ruby, which was set up in our Dockerfile above.

Use the next two keys: tty and stdin_open, for debugging with binding.pry while hitting the Rails server. Check out this gist for more info.

One thing to call out from that gist is to use ctrl-p + ctrl-q to detach from the Docker container and leave it in a running state. The depends_on key is used to declare other containers that are required to start with the service. Currently, it is only dependent on postgres. No other health checks or validations are run; we will handle those in the entrypoint.

The Postgres Service

  services:
    rails:
      . . .
    postgres:
      image: postgres:11
      environment:
        - POSTGRES_HOST_AUTH_METHOD=trust
      volumes:
        - postgres:/var/lib/postgresql/data

We define our postgres service next with pretty minimal configuration compared to what we went through for the rails service.

The first key, image, will check locally first, then download the official postgres image from Docker Hub with a matching tag if needed. In this case, it will pull in 11.5. I chose 11 for this example because that is now the default on Heroku, but there are lots of image options available on Docker Hub.

We are using a named volume here as well for our postgres data.

And, that wraps up the Compose file.

Enter The Entrypoint

There are a few requirements for starting a Rails server, e.g., the database running and accepting connections. We use the entrypoint, which is a bash script, to fulfill those requirements. Looking back at the compose file we should create this bash script at docker/ruby/entrypoint.sh and make it executable.

  #! /bin/bash
  set -e

  : ${APP_PATH:="/app"}
  : ${APP_TEMP_PATH:="$APP_PATH/tmp"}
  : ${APP_SETUP_LOCK:="$APP_TEMP_PATH/setup.lock"}
  : ${APP_SETUP_WAIT:="5"}

  # 1: Define the functions to lock and unlock our app container's setup
  # processes:
  function lock_setup { mkdir -p $APP_TEMP_PATH && touch $APP_SETUP_LOCK; }
  function unlock_setup { rm -rf $APP_SETUP_LOCK; }
  function wait_setup { echo "Waiting for app setup to finish..."; sleep $APP_SETUP_WAIT; }

  # 2: 'Unlock' the setup process if the script exits prematurely:
  trap unlock_setup HUP INT QUIT KILL TERM EXIT

  # 3: Wait for postgres to come up
  echo "DB is not ready, sleeping..."
  until nc -vz postgres 5432 &>/dev/null; do
    sleep 1
  done
  echo "DB is ready, starting Rails."

  # 4: Specify a default command, in case it wasn't issued:
  if [ -z "$1" ]; then set -- bundle exec rails server -p 3000 -b 0.0.0.0 "$@"; fi

  # 5: Run the checks only if the app code is executed:
  if [[ "$3" = "rails" ]]
  then
    # Clean up any orphaned lock file
    unlock_setup
    # 6: Wait until the setup 'lock' file no longer exists:
    while [ -f $APP_SETUP_LOCK ]; do wait_setup; done

    # 7: 'Lock' the setup process, to prevent a race condition when the
    # project's app containers will try to install gems and set up the
    # database concurrently:
    lock_setup
    # 8: Check if dependencies need to be installed and install them
    bundle install

    yarn install
    # 9: Run migrations or set up the database if it doesn't exist
    # Rails >= 6
    bundle exec rails db:prepare
    # Rails < 6
    # bundle exec rake db:migrate 2>/dev/null || bundle exec rake db:setup

    # 10: 'Unlock' the setup process:
    unlock_setup

    # 11: If the command to execute is 'rails server', then we must remove any
    # pid file present. Suddenly killing and removing app containers might leave
    # this file, and prevent rails from starting-up if present:
    if [[ "$4" = "s" || "$4" = "server" ]]; then rm -rf /app/tmp/pids/server.pid; fi
  fi
  # 12: Replace the shell with the given command:
  exec "$@"

I will only call out a few things specific to working with Rails from this file. Below, we are using nc(netcat) to verify that postgres is up before running any database commands. nc doesn't just ping the server; it is checking that the service is responding on a specific port. This check runs every second until it boots. Note: We are using the service name postgres to connect to the container.

  echo "DB is not ready, sleeping..."
  until nc -vz postgres 5432 &>/dev/null; do
    sleep 1
  done
  echo "DB is ready, starting Rails."

Next, we run our bundle and yarn install commands to make sure we have the latest dependencies. Once both of those have run, we will use the new Rails 6 db:prepare method to either set up the database or run migrations. This can be handled in many different ways but my preference is to use Rails tools. (Note: the comment is how you would do it on Rails 5 or older)

  bundle install

  yarn install
    # Rails >= 6
    bundle exec rails db:prepare
    # Rails < 6
    # bundle exec rake db:migrate 2>/dev/null || bundle exec rake db:setup

Lastly, we check for any dangling PID files left from killing and removing the app containers. If the PID files are left our rails server command will fail.

  if [[ "$4" = "s" || "$4" = "server" ]]; then rm -rf /app/tmp/pids/server.pid; fi

Start It Up

With all of this in place, we run docker-compose up, which will build and start our containers. It is also a great time to get coffee because building the app container will take some time. Once everything is built and up, point your browser at localhost:3000 and your homepage, or if this is a new app, it will display the default Rails homepage. If you need to run specs or any other Rails command, running docker-compose exec app bash in a separate terminal will launch a Bash session on the running app container.

Wrapping Up

We now have a base-level Rails Docker development environment. Here is a gist of all the three files referenced in this guide. Now that you have an understanding of how to set up a Docker development environment, check out Railsdock! It is a CLI tool we are working on that will generate most of this config for you. PR's appreciated!

In the next part of the series, we will set up more services and use Compose features to share config between similar services.