What is Compose Include and What problem does it solve?

Docker Compose is a powerful tool for defining and running multi-container Docker applications. It enables you to manage complex applications with multiple services, networks, and volumes using a simple YAML file. However, as applications grow in complexity, the Docker Compose file can become lengthy and harder to manage. This is where the docker-compose.yml file's include directive comes into play. In this blog, we will explore Docker Compose include, its purpose, and how it solves the problem of managing large and intricate Compose files.

Compose Include is a new feature available now in Compose v2.20.0 and starting Docker Desktop 4.22 release. It allows you to include the contents of another file in your compose file. This can be useful for sharing common configuration or environment variables between multiple services.

The Challenge: Complexity in Docker Compose Files

Modern applications often consist of multiple services, databases, caching systems, and more. As the number of services grows, the Docker Compose file can become quite extensive and challenging to maintain. For example, imagine an application that includes a web service, a database, a caching server, and a message queue. The docker-compose.yml file for such an application could quickly become cluttered and hard to read, even with proper indentation and formatting.

Benefits of Docker Compose Include

• Modularity: By splitting your Docker Compose configuration into smaller files, you can focus on individual services or components, making it easier to understand and maintain.

• Reusability: You can reuse configuration files across different projects or environments, saving time and effort in setting up similar services.

• Collaboration: Different team members can work on separate service configurations simultaneously without conflicting with each other's changes.

• Readability: Smaller configuration files are more readable and make it easier to locate specific settings or services.

• Simplification: Docker Compose include helps avoid unnecessary repetition of configuration settings, reducing the risk of errors.

Why use Compose Include?

There are a few reasons why you might want to use Compose Include:

• To share common configuration

If you have a set of configuration that you use in multiple services, you can include it in a single file and then include that file in each of your compose files. This can help to keep your compose files DRY (Don't Repeat Yourself).

• To share environment variables

You can also use Compose Include to share environment variables between multiple services. This can be useful for things like setting the database connection string or the API key.

• To make your compose files more readable

If you have a lot of configuration or environment variables in your compose files, it can be difficult to keep track of everything. By including these in separate files, you can make your compose files easier to read and understand.

How Docker Compose Include Works

The docker-compose.yml file that uses the include directive will reference and import configurations from other YAML files. This allows you to maintain separate files for different services or components of your application. The main docker-compose.yml file then serves as an orchestrator, pulling in the various pieces through the include directive.

How to use Compose Include

To use Compose Include, you need to add the include keyword to your compose file. The syntax is as follows:

include: <file>

where is the path to the file that you want to include.

For example, the following compose file includes a file called common.yaml:

version: '3.8'
services:
  web:
    image: nginx
    ports:
      - 80:80
    include: common.yaml

The common.yaml file could contain the following configuration:

environment:
  DATABASE_URL: postgres://localhost:5432/my_database
  API_KEY: 1234567890

This would make the DATABASE_URL and API_KEY environment variables available to the web service.

Getting Started

Pre-requisite

• Install Docker Compose v2.20.x

wget https://github.com/docker/compose/releases/download/v2.20.2/docker-compose-linux-x86_64
mv docker-compose-linux-x86_64 docker-compose
chmod +x docker-compose

docker-compose version
Docker Compose version v2.20.2

Clone the repository

git clone https://github.com/ajeetraina/compose-include/
cd compose-include

Viewing the content of web.yaml

version: '2'
include:
  - postgresql.yaml
services:
  web:
    image: nginx:latest
    depends_on:
      - database

Here's a snippet of a docker-compose.yml file that includes a PostgreSQL service configuration and a web service configuration that depends on the database. This setup suggests you're working with a web application that utilizes both Nginx as a web server and PostgreSQL as a database.

Here's a breakdown of the provided code:

• Version Declaration The version field indicates the version of the Docker Compose file format being used. In this case, you're using version '2'.

Include Directive

The include directive allows you to split your Docker Compose configuration into multiple files. In your case, you are including the configuration from postgresql.yaml.

Service Configuration - PostgreSQL

The configuration for the PostgreSQL service is defined in the postgresql.yaml file (not shown in your snippet). This could include settings like image, environment variables, volumes, and more, specific to the PostgreSQL service.

Service Configuration - Web (Nginx)

The web service is configured to use the Nginx Docker image (nginx:latest). It also has a dependency on the database service, which is defined in the postgresql.yaml file. This means that the web service won't start until the database service is up and running.

Let's look at the posgresql.yaml file

services:
  database:
    image: postgres:12
    environment:
      POSTGRES_PASSWORD: password

Here's a breakdown of the provided code:

Service Configuration - Database (PostgreSQL)

The service is named database. It uses the Docker image postgres:12, which refers to PostgreSQL version 12.

The environment section

It is used to set environment variables for the PostgreSQL container. In this case, you've set the POSTGRES_PASSWORD environment variable to "password". This password is used to authenticate with the PostgreSQL server.

This configuration sets up a PostgreSQL container using the specified image and environment variables. The POSTGRES_PASSWORD environment variable ensures that the PostgreSQL instance is secured with the provided password.

Run the Services

docker-compose -f web.yaml up

docker-compose -f web.yaml ps 
NAME                IMAGE               COMMAND                  SERVICE             CREATED             STATUS              PORTS
root-database-1     postgres:12         "docker-entrypoint.s…"   database            10 minutes ago      Up 2 minutes        5432/tcp
root-web-1          nginx:latest        "/docker-entrypoint.…"   web                 10 minutes ago      Up 2 minutes        80/tcp

Conclusion

Compose Include is a powerful feature that can help you to make your Docker Compose files more readable and maintainable. By including common configuration and environment variables in separate files, you can keep your compose files DRY and make it easier to share them with others.

Docker Compose include is a valuable feature that simplifies the management of complex multi-service deployments. By breaking down your Docker Compose configuration into smaller, modular files, you can enhance readability, reusability, collaboration, and overall maintainability of your applications. Embrace Docker Compose include to make your Dockerized application development and deployment smoother and more organized.