Docker is an in-demand, DevOps technology sought after in the tech industry. This powerful tool allows you to set up and deploy applications using containers. With the consistent environment of Docker, the development lifecycle of applications is easy and seamless. Docker Compose, a more advanced Docker tool, can be used to simplify your workflow. In this article, we will refresh your knowledge of Docker and show you how to get started with Docker Compose. To be most successful with this article, you should already have some experience with Docker.
If you're new to DevOps, check out our beginner's guide to Docker and Kubernetes before proceeding with this tutorial.
Today, we will go over:
- Brief refresher on Docker
- Fundamentals of Docker
- Getting started with Docker Compose
- What to learn next
- Wrapping up and resources
Docker is an open-source containerization tool used to simplify the creation and deployment of applications by using the concept of containers. Containers allow us to package all the parts of an application and deploy it as one entity. This tool makes it easy for different developers to work on the same project in the same environment without any dependencies or OS issues. Docker is sort of like a virtual machine, but Docker enables applications to access the same Linux kernel. Docker offers many advantages for developers and DevOps teams. Docker...
- is highly demanded by companies large and small
- offers isolation from the main system
- simplifies configuration
- provides access to thousands of configured images with Docker Hub
- supports many CI tools like Travis and Jenkins
- allows developers to focus just on writing code
- simplifies deployment management for operations teams
Docker is commonly used alongside Kubernetes, a powerful container management tool that automates the deployment of your Docker containers. While Docker is used to isolate, pack, and ship your application into containers, Kubernetes is like the container scheduler for deploying and scaling the application. The two technologies are designed to work together and make app deployment a breeze.
Before diving into advanced Docker concepts, like Docker Compose, we want to make sure to refresh the fundamentals of Docker as a whole. Let's define and explore the basics of Docker.
The Docker Architecture is made of layers, as we will discuss below. The bottom layer is the physical server that we use to host virtual machines. This is the same as a traditional virtualization architecture. The second layer is the Host OS, which is the base machine (i.e. Windows or Linux). Next, is the Docker Engine, which we use to run the operating system. Above that is are the Apps which run as Docker containers. Those Docker Objects are made up of images and containers.
The basic structure of Docker relies on images and containers. Think of images and containers as two different states of the same underlying concept. A container is like an object, and an image is like its class. Think of a container as an isolated system that contains everything needed to run a certain application. It is an instance of an image that simulates the required environment. Below is an example command when we run a ubuntu Docker container:
docker run -i -t ubuntu /bin/bash
Images, on the other hand, are used to start-up containers. From running containers, we can get images, which can be composed together to form a system-agnostic way of packaging applications. Images can be pre-built, retrieved from registries, created from already existing ones, or combined together via a common network.
Dockerfiles are how we containerize our application, or how we build a new container from an already pre-built image and add custom logic to start our application. From a Dockerfile, we use the Docker build command to create an image. Think of a Dockerfile as a text document that contains the commands we call on the command line to build an image. Below is an example of a Dockerfile:
FROM python:3 WORKDIR /usr/src/app COPY requirements.txt ./ RUN pip install --no-cache-dir -r requirements.txt COPY . . CMD [ "python", "./your-daemon-or-script.py" ]
A Dockerfile works in layers. These are the building blocks of Docker. The first layer starts with the
FROMkeyword and defines which pre-built image we will use build an image. We can then define user permissions and startup scripts. In Docker, a container is an image with a readable layer build on top of a read-only layers. These layer are called intermediate images, and they are generated when we execute the commands in our Dockerfile during the build stage.
Docker Hub is a Docker Registry that provides unlimited storage of public images and paid plans for hosting private images. A public image may be accessed by anyone. You can publish and access images on Docker Hub once you make an account. You can do this using the docker build command or docker tags to generate an image ID. You can publish an image on Docker Hub using the following commands:
docker login docker push learnbook/webserver
That was just an overview of the fundamentals of Docker before we move onto move advanced concepts. Keep in mind that there's a lot more to Docker than what we discussed above. If you want to continue learning about Docker before advancing, I recommend the beginner's course Docker for Developers, which will walk you through these basics.
Now for the advanced stuff. Docker Compose is a Docker tool used to define and run multi-container applications. With Compose, you use a
YAML file to configure your application’s services and create all the app's services from that configuration. Think of
docker-compose as an automated multi-container workflow. Compose is an excellent tool for development, testing, CI workflows, and staging environments. According to the Docker documentation, the most popular features of Docker Compose are:
- Multiple isolated environments on a single host
- Preserve volume data when containers are created
- Only recreate containers that have changed
- Variables and moving a composition between environments
- Orchestrate multiple containers that work together
Compose uses the Docker Engine, so you'll need to have the Docker Engine installed on your device. You can run Compose on Windows, Mac, and 64-bit Linux. Installing Docker Compose is actually quite easy. On desktop systems, such as Docker Desktop for Mac and Windows, Docker Compose is already included. No additional steps are needed. On Linux systems, you'll need to:
- Install the Docker Engine
- Run the following command to download Docker Compose
sudo curl -L "https://github.com/docker/compose/releases/download/1.26.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
- Apply permissions to the binary, like so:
sudo chmod +x /usr/local/bin/docker-compose
- Test the installation to check it worked properly
$ docker-compose --version docker-compose version 1.26.2, build 1110ad01
For more details on installing Docker and Compose, visit the official Docker documentation page.
Regardless of how you chose to install it, once you have Docker Compose downloaded and running properly, you can start using it with your Dockerfiles. This process requires three basic steps:
- Define your app's environment using a Dockerfile. This way, it can be reproduced.
- Define the services for your app in a
docker-compose.ymlfile. This way, they can run in an isolated environment.
docker-composeto start your app.
You can easily add Docker Compose to a pre-existing project. If you already have some Dockerfiles, add Docker Compose files by opening the Command Palette. Use the Docker: Docker Compose Files to the Workspace command, and, when promoted, choose the Dockerfiles you want to include.
You can also add Docker Compose files to your workspace when you add a Dockerfile. Similarly, open the Command Palette and use the Docker: Add Docker Files to Workspace command. You'll then be asked if you want to add any Docker Compose files. In both cases, Compose extension will add the
docker-compose.yml file to your workspace.
Now that we know how to download Docker Compose, we need to understand how Compose files work. It's actually simpler than it seems. In short, Docker Compose files work by applying mutiple commands that are declared within a single
docker-compose.yml configuration file. The basic structure of a Docker Compose YAML file looks like this:
version: 'X' services: web: build: . ports: - "5000:5000" volumes: - .:/code redis: image: redis
Now, let's look at real-world example of a Docker Compose file and break it down step-by-step to understand all of this better. Note that all the clauses and keywords in this example are commonly used keywords and industry standard. With just these, you can start a development workflow. There are some more advanced keywords that you can use in production, but for now, let's just get started with the necessary clauses.
version: '3' services: web: # Path to dockerfile. # '.' represents the current directory in which # docker-compose.yml is present. build: . # Mapping of container port to host ports: - "5000:5000" # Mount volume volumes: - "/usercode/:/code" # Link database container to app container # for reachability. links: - "database:backenddb" database: # image to fetch from docker hub image: mysql/mysql-server:5.7 # Environment variables for startup script # container will use these variables # to start the container with these define variables. environment: - "MYSQL_ROOT_PASSWORD=root" - "MYSQL_USER=testuser" - "MYSQL_PASSWORD=admin123" - "MYSQL_DATABASE=backend" # Mount init.sql file to automatically run # and create tables for us. # everything in docker-entrypoint-initdb.d folder # is executed as soon as container is up nd running. volumes: - "/usercode/db/init.sql:/docker-entrypoint-initdb.d/init.sql"
version ‘3’: This denotes that we are using version 3 of Docker Compose, and Docker will provide the appropriate features. At the time of writing this article, version 3.7 is latest version of Compose.
services: This section defines all the different containers we will create. In our example, we have two services, web and database.
web: This is the name of our Flask app service. Docker Compose will create containers with the name we provide.
build: This specifies the location of our Dockerfile, and
.represents the directory where the
docker-compose.ymlfile is located.
ports: This is used to map the container's ports to the host machine.
volumes: This is just like the
-voption for mounting disks in Docker. In this example, we attach our code files directory to the containers'
./codedirectory. This way, we won't have to rebuild the images if changes are made.
links: This will link one service to another. For the bridge network, we must specify which container should be accessible to which container using links.
image: If we don’t have a Dockerfile and want to run a service using a pre-built image, we specify the image location using the
imageclause. Compose will fork a container from that image.
environment: The clause allows us to set up an environment variable in the container. This is the same as the
-eargument in Docker when running a container.
Congrats! Now you know a bit about Docker Compose and the necessary parts you'll need to get started with your workflow.
Learn advanced Docker and Docker Compose without scrubbing through videos or documentation. Educative's text-based courses are easy to skim and feature live coding environments, making learning quick and efficient.
Now that we know how to create a
docker-compose file, let's go over the most common Docker Compose commands that we can use with our files. Keep in mind that we will only be discussing the most frequently-used commands.
docker-compose: Every Compose command starts with this command. You can also use
docker-compose <command> --help to provide additional information about arguments and implementation details.
$ docker-compose --help Define and run multi-container applications with Docker.
docker-compose build: This command builds images in the
docker-compose.yml file. The job of the
build command is to get the images ready to create containers, so if a service is using the prebuilt image, it will skip this service.
$ docker-compose build database uses an image, skipping Building web Step 1/11 : FROM python:3.9-rc-buster ---> 2e0edf7d3a8a Step 2/11 : RUN apt-get update && apt-get install -y docker.io
docker-compose images: This command will list the images you've built using the current
$ docker-compose images Container Repository Tag Image Id Size -------------------------------------------------------------------------------------- 7001788f31a9_docker_database_1 mysql/mysql-server 5.7 2a6c84ecfcb2 333.9 MB docker_database_1 mysql/mysql-server 5.7 2a6c84ecfcb2 333.9 MB docker_web_1 <none> <none> d986d824dae4 953 MB
docker-compose stop: This command stops the running containers of specified services.
$ docker-compose stop Stopping docker_web_1 ... done Stopping docker_database_1 ... done
docker-compose run: This is similar to the
docker run command. It will create containers from images built for the services mentioned in the compose file.
$ docker-compose run web Starting 7001788f31a9_docker_database_1 ... done * Serving Flask app "app.py" (lazy loading) * Environment: development * Debug mode: on * Running on http://0.0.0.0:5000/ (Press CTRL+C to quit) * Restarting with stat * Debugger is active! * Debugger PIN: 116-917-688
docker-compose up: This command does the work of the
docker-compose build and
docker-compose run commands. It builds the images if they are not located locally and starts the containers. If images are already built, it will fork the container directly.
$ docker-compose up Creating docker_database_1 ... done Creating docker_web_1 ... done Attaching to docker_database_1, docker_web_1
docker-compose ps: This command list all the containers in the current
docker-compose file. They can then either be running or stopped.
$ docker-compose ps Name Command State Ports --------------------------------------------------------------------------------- docker_database_1 /entrypoint.sh mysqld Up (healthy) 3306/tcp, 33060/tcp docker_web_1 flask run Up 0.0.0.0:5000->5000/tcp $ docker-compose ps Name Command State Ports ---------------------------------------------------------- docker_database_1 /entrypoint.sh mysqld Exit 0 docker_web_1 flask run Exit 0
docker-compose down: This command is similar to the
docker system prune command. However, in Compose, it stops all the services and cleans up the containers, networks, and images.
$ docker-compose down Removing docker_web_1 ... done Removing docker_database_1 ... done Removing network docker_default (django-tuts) Venkateshs-MacBook-Air:Docker venkateshachintalwar$ docker-compose images Container Repository Tag Image Id Size ---------------------------------------------- (django-tuts) Venkateshs-MacBook-Air:Docker venkateshachintalwar$ docker-compose ps Name Command State Ports ------------------------------
Congrats! You've now learned most of the basic commands for Docker Compose. Why stop there? Check out the documentation of other commands and keep learning!
I hope this has familiarized you with Docker Compose and all it has to offer. There's still a lot to explore and learn to be a true Docker Compose master. Once you are comfortable making
docker-compose files and working with the necessary commands, you can move onto the following advancements:
- Working with multiple Dockerfiles in Compose (common for microservices)
- Docker Compose environment variables (
- Docker Swarm (for scaling and monitoring clusters)
- Automated deployments with Docker Stack
- and more
Congrats on getting this far with Docker Compose. Today, we walked you through a Docker refresher and introduced you to the basics of Docker Compose, including how to create a file and all the need-to-know Compose commands. Docker is a notoriously challenging technology to learn, but the skills you'll learn will make you an in-demand, modern developer. Keep learning, reading, and advancing your Docker skills!