Docker includes Swarm mode for natively managing a cluster of Docker Engines called a swarm.
There is excellent tutorial Getting started with Swarm mode, this tutorial describes how to create docker swarm cluster with 1 manager and 2 worker nodes.
This article shows how to setup high-availability (HA) Docker Swarm cluster using Vagrant.
Preconditions
- You have a computer with at least 12 GB of free RAM
- You have installed Vagrant on this computer
- Install a virtualization product such as VirtualBox
Setup
Initialize a project directory
Make a new directory for the project you will work:
$ mkdir docker-swarm-ha
$ cd docker-swarm-ha
Inventory file
Create an inventory.yaml file in your current directory, and open the file in you favorite editor:
$ touch inventory.yaml
# vim inventory.yaml
Fill the file like this one:
---
- role: Manager
availability: drain
hostname: manager-1
ip: 192.168.100.10
image: centos/8
ram: 2048
cpus: 1
- role: Manager
availability: drain
hostname: manager-2
ip: 192.168.100.11
image: centos/8
ram: 2048
cpus: 1
- role: Manager
availability: drain
hostname: manager-3
ip: 192.168.100.12
image: centos/8
ram: 2048
cpus: 1
- role: Worker
availability: active
hostname: worker-1
ip: 192.168.100.13
image: centos/8
ram: 4096
cpus: 1
- role: Worker
availability: active
hostname: worker-2
ip: 192.168.100.14
image: centos/8
ram: 4096
cpus: 1
- role: Worker
availability: active
hostname: worker-3
ip: 192.168.100.15
image: centos/8
ram: 4096
cpus: 1
This is inventory file of your docker swarm cluster with relevant virtual machine (VM) characteristics.
Vagrant file
Create a Vagrantfile in your current directory, and open the file in you favorite editor:
$ touch Vagrantfile
# code .
Now, it's time to code Vagrantfile using Ruby language...
On top of the file add following code
# -*- mode: ruby -*-
# vi: set ft=ruby :
require 'yaml'
current_dir = File.dirname(File.expand_path(__FILE__))
servers = YAML.load_file("#{current_dir}/inventory.yaml")
leader = servers.find { |favor| favor['hostname'] == 'manager-1' }
group = "Docker Swarm"
Here, we load the inventory file, and then we define a leader node in the cluster. This node is used to initialize docker swarm.
Also we defined Docker Swarm as a group of VMs in VirtualBox GUI.
Add Vagrant configuration below
Vagrant.configure("2") do |config|
end
The "2" in Vagrant.configure configures the configuration version.
Inside this block is the place where the main code will be placed.
Define all VMs in accordance with inventory file
servers.each do |machine|
config.vm.define machine['hostname'] do |node|
node.vm.box = machine['image']
node.vm.hostname = machine['hostname']
node.vm.network :private_network, ip: machine['ip']
node.vm.provider :virtualbox do |vb|
vb.customize ["modifyvm", :id, "--groups", "/#{group}/#{machine['role']}"]
vb.name = machine['hostname']
vb.memory = machine['ram']
vb.cpus = machine['cpus']
end
end
end
Define hosts file on all VMs
servers.each do |machine|
config.vm.provision "Setup hosts", type: :shell, :args => [machine['ip'], machine['hostname']], inline: <<-SHELL
sudo echo "$1 $2" >> /etc/hosts
SHELL
end
Setup Linux package repositories
config.vm.provision "Setup repositories", type: :shell, privileged: true, inline: <<-SHELL
sed -i s/mirror.centos.org/vault.centos.org/g /etc/yum.repos.d/CentOS-*.repo
sed -i s/^mirrorlist=http/#mirrorlist=http/g /etc/yum.repos.d/CentOS-*.repo
sed -i s/^#.*baseurl=http/baseurl=http/g /etc/yum.repos.d/CentOS-*.repo
yum clean all
SHELL
Install Docker Engine on all VMs
config.vm.provision "Install Docker", type: :docker
Install Redis as a cache in docker container
config.vm.provision "Run Redis cache", type: :docker do |d|
d.run "redis",
cmd: "--bind 0.0.0.0",
args: "-p 0.0.0.0:6379:6379"
end
In our case, Redis cache is used to temporary store docker swarm join tokens.
Initialize docker swarm
config.vm.define "manager-1" do |node|
node.vm.provision "Swarm init", type: :shell, privileged: true, :args => [leader['ip'], leader['availability']],
inline: "docker swarm init --advertise-addr $1 --availability $2 || true"
node.vm.provision "Save join-tokens", type: :shell, privileged: true,
inline: <<-SHELL
docker exec redis redis-cli set manager-join-token $(docker swarm join-token manager -q)
docker exec redis redis-cli set worker-join-token $(docker swarm join-token worker -q)
SHELL
end
As you can see above, join tokens are saved into Redis cache.
And finally join all VMs to the docker swarm cluster
servers.select { |favor| favor['hostname'] != 'manager-1' }.each do |machine|
config.vm.define machine['hostname'] do |node|
node.vm.provision "Add a #{machine['role']} to the swarm", type: :shell, privileged: true,
:args => [leader['ip'], machine['role'].downcase, machine['availability']],
inline: <<-SHELL
token=$(docker exec redis redis-cli -h $1 get $2-join-token)
docker swarm join --availability $3 --token $token $1 || true
SHELL
end
end
Boot an environment
Bring up a virtual machine
Run the following from your terminal:
$ vagrant up
This command will finish and you will have VMs running with docker swarm.
Check swarm nodes
Connect to a master node using SSH:
$ vagrant ssh manager-1
[vagrant@manager-1 ~]$
Finally, verify the status of the cluster nodes using the following command:
[vagrant@manager-1 ~]$ docker node ls
ID HOSTNAME STATUS AVAILABILITY MANAGER STATUS ENGINE VERSION
illxi93is8446ghgl86lihg1f * manager-1 Ready Drain Leader 26.1.3
es0ktcmq5d9403p16b688knat manager-2 Ready Drain Reachable 26.1.3
j1wnwxnysm763eex9r024h8if manager-3 Ready Drain Reachable 26.1.3
comemxromzh4y8a6ciou6smk8 worker-1 Ready Active 26.1.3
smz5sf6vdbzszc850zxwddfie worker-2 Ready Active 26.1.3
p7r1khx4h8famoi9fb0itx8kg worker-3 Ready Active 26.1.3
VirualBox GUI
Now VirualBox GUI should looks like this:
Deploy a service
In order to verify that everything is OK, we will deploy the simplest docker service to a swarm:
$ docker service create --name my_web \
--replicas 3 \
--publish published=8080,target=80 \
nginx:latest
The service is scheduled on available nodes. To confirm that the service was created and started successfully, use the docker service ls command:
[vagrant@manager-1 ~]$ docker service create --name my_web \
> --replicas 3 \
> --publish published=8080,target=80 \
> nginx:latest
s3vgnoaca0p14jgd27ag52th4
overall progress: 3 out of 3 tasks
1/3: running [==================================================>]
2/3: running [==================================================>]
3/3: running [==================================================>]
verify: Service s3vgnoaca0p14jgd27ag52th4 converged
[vagrant@manager-1 ~]$ docker service ls
ID NAME MODE REPLICAS IMAGE PORTS
s3vgnoaca0p1 my_web replicated 3/3 nginx:latest *:8080->80/tcp
Now, you may open just deployed nginx service in web browser using one of the VM's IP:
Cleanup
To delete docker service, use the docker service rm command:
[vagrant@manager-1 ~]$ docker service rm my_web
my_web
[vagrant@manager-1 ~]$ docker service ls
ID NAME MODE REPLICAS IMAGE PORTS
To stop VMs that Vagrant is managing and remove all the resources created during the machine-creation process, use the vagrant destroy command:
$ vagrant destroy
worker-3: Are you sure you want to destroy the 'worker-3' VM? [y/N]
To gracefully shut down the VMs, use the vagrant halt command:
$ vagrant halt
==> worker-3: Attempting graceful shutdown of VM...
==> worker-2: Attempting graceful shutdown of VM...
==> worker-2: Forcing shutdown of VM...
==> worker-1: Attempting graceful shutdown of VM...
==> manager-3: Attempting graceful shutdown of VM...
==> manager-2: Attempting graceful shutdown of VM...
==> manager-1: Attempting graceful shutdown of VM...
Summary
In this post, we’ve seen:
- how to easily setup high-availability (HA) Docker Swarm cluster using Vagrant
- how to deploy docker service to a swarm cluster
- how to clean up environment.
The codebase for this article can be found here.
Top comments (0)