Dockerizing a JVM app with CircleCI

I had been using Travis for a long time for my personal projects, until the news of massive layoffs scared me into checking alternatives. I had heard good things about CircleCI, so I decided to give it a try. Concretely, I wanted to build a pipeline that did the following:

• Package a SpringBoot application written in Kotlin as a jar, running linters and tests before, saving the tests results.
• Build a Docker image to run the app, with ServerSpec running first, and push it to a Docker registry.

Following my own thoughts about pipelines, I wanted it to be fast, visual, easy to maintain and reliable build my toy app. Did it work? More after the break!

Running in circles

CircleCI has a free plan that works really well for experimentation, with a seamless integration with Github, which makes getting started a breeze.

The basics are covered in depth in their own website, so I will skip that. Let’s jump straight into the details. This is the sketch of the pipeline to implement:

I’m going to talk about some details that I found interesting, plus the code to implement parts of the pipeline. TLDR: This is the full pipeline.

Using workflows

Workflows are CircleCI’s way of organizing dependencies between jobs. In my case that’s just a fancy way of saying that one job should execute after the other, like this screenshot shows:

Looks simple, right? However, the picture does not show the whole truth. I am using a filter to restrict the docker job to the master branch. The workflow definition looks like this:

workflows:
  version: 2
  pipeline:
    jobs:
      - build
      - docker:
          filters:
            branches:
              only: master
          requires:
            - build

Why the restriction? Well, I’m using Dependabot to automatically check for new versions of dependencies, and I want the PRs generated by the tool to run my tests, but not to generate new docker images.

The build step

I always run my jobs inside some Docker image. For the build I use the OpenJDK image provided by CircleCI, which is one among the many images that they maintain. These are the steps.

build:
  docker:
    - image: circleci/openjdk:8-jdk

  steps:
    - checkout
    - run: ./gradlew detekt
    - run: ./gradlew clean test
    - run: ./gradlew assemble

detekt is basically a linter that can run self-contained inside the pipeline. Apart from that, the unit tests are executed and if everything works, a jar is built.

Persistence

Unlike Concourse, CircleCI allows you to persist artifacts between jobs, which is pretty damn convenient. In this case, I need the jar file that we just generated later, when I build the Docker image. In order to persist it, you add an extra directive to the list of steps in the build job:

- persist_to_workspace:
    root: build/libs
    paths:
      - echo.jar

You’ve got to save the tests

Another nifty feature that I found is storing test results, which appear in the dashboard and make it easier to understand what your job is doing.

Both the test results, as well as the execution summary are stored as artifacts. For that there are some additional steps to add:

- run:
    name: Save test results
    command: |
      mkdir -p ~/junit/
      find . -type f -regex ".*/build/test-results/.*xml" -exec cp {} ~/junit/ \;
    when: always

- store_test_results:
    path: ~/junit
- store_artifacts:
    path: ~/junit

All in all the build has a bunch of steps, as can be seen in this screenshot.

Docker step

The image itself

So if I want to dockerize our app I kind of need a Dockerfile (duh!). This image runs the jar that was built on the previous job:

FROM openjdk:8-jre-alpine3.9

WORKDIR /app
EXPOSE 4000

RUN apk add --update --no-cache dumb-init \
  && rm -rf /var/cache/apk/*

COPY build/libs/echo.jar .

RUN adduser -D runner

USER runner
ENTRYPOINT ["/usr/bin/dumb-init", "--"]
CMD ["java", "-jar", "echo.jar"]

Nothing out of the ordinary. Here is a recent post about improving the quality of your Docker images.

Running ServerSpec

I am a big fan of testing containers with ServerSpec, as I’ve written before and spoken about multiple times.

Running these tests in a CI tends to be painful because of the infamous docker in docker problem. I was expecting yet another round of pain when trying to set them up for CircleCI.

Not so much! Surprisingly, it went quite swiftly. Maybe I’m just used to it by now, who knows. How to do it, though?

I’ve set up a new job for this. I created and uploaded my own dind with ruby image so that I have access to newer ruby versions. A simplified version of it looks like this:

docker:

  docker:
    - image: sirech/dind-ruby:2.6.3

  steps:

    - checkout
    - attach_workspace:
        at: build/libs
    - setup_remote_docker
    - run: docker build . -t echo
    - run:
        name: ServerSpec Tests
        command: |
          bundle install
          bundle exec rspec spec

Thanks to the setup_remote_docker directive, regular docker commands are available for this job. After installing the dependencies, the tests can run inside the pipeline.

The jar that was generated in the previous job is available by using the attach_workspace directive.

Building the image

Once the container tests have passed, it’s time to build the image and publish it to some registry. That will be our final artifact that can be used for deployment. Deployment can happen in the same pipeline or in a separate one, it doesn’t really matter.

Starting from the same job as before, it’s time to log in to whatever registry we should hold the image. Then we build the image with the right tag, and push it:

- run:
    name: Publish docker image
    command: |
      echo $DOCKER_PASSWORD | docker login -u $DOCKER_USER $DOCKER_REGISTRY --password-stdin
      docker build . --tag $DOCKER_REGISTRY/echo
      docker push $DOCKER_REGISTRY/echo

You see that I’m not versioning the image. Don’t do this for a serious application. You can attach the git commit by using the CIRCLE_SHA1 variable, which is one of the many built-in variables that are available out of the box.

Caching

Isolation is great at keeping our builds reliable. It is often at odds with performance, though. I don’t want to cache the whole workspace like in the old, dark Jenkins days, but avoiding downloading the dependencies every time is quite convenient and keeps the pipeline fast, just the way I like it.

In CircleCI, you do that by saving a particular folder based on a key. For instance, if I want to cache the ruby dependencies that I use for the container tests, I use a block like this:

- restore_cache:
    key: bundle-2-{{ checksum "Gemfile.lock" }}
- run: bundle check || bundle install --without ops development
- save_cache:
    key: bundle-2-{{ checksum "Gemfile.lock" }}
    paths:
      - vendor/bundle

All the steps inside the same job after this will have them available, and they will be cached until there is a change to the Gemfile.lock file.

Defining variables

I have been using variables throughout my pipeline to encode certain values, like my Docker credentials. They are managed through the UI, which frankly is not ideal. I would prefer a more programmatic solution, to be honest. Needless to say, putting secrets in the config.yml directly is out of the question, so this will have to do.

If you want to manage a set of variables for multiple projects, one option could be using contexts.

How does it all look together?

That wasn’t that bad after all. I find CircleCI quite intuitive, and for hobby projects their standard account is more than enough. In fact, the pipelines run very quickly. In the beginning I was waiting a bit to check on them, and they had been finished for a while by the time I had a look.

The whole pipeline is in github. That should be a good start to build your own app. In fact, the pipeline does not stop at the Docker stage, but has a deployment step based on a custom script.