DEV Community

Ježek
Ježek

Posted on • Edited on

Setup a Docker Swarm cluster using Vagrant

#docker #swarn #vagrant

Docker includes Swarm mode for natively managing a cluster of Docker Engines called a swarm.

There is excellent tutorial Getting started with Swarm mode, this tutorial describes how to create docker swarm cluster with 1 manager and 2 worker nodes.

This article shows how to setup high-availability (HA) Docker Swarm cluster using Vagrant.

Preconditions

  • You have a computer with at least 12 GB of free RAM
  • You have installed Vagrant on this computer
  • Install a virtualization product such as VirtualBox

Setup

Initialize a project directory

Make a new directory for the project you will work:

$ mkdir docker-swarm-ha
$ cd docker-swarm-ha
Enter fullscreen mode Exit fullscreen mode

Inventory file

Create an inventory.yaml file in your current directory, and open the file in you favorite editor:

$ touch inventory.yaml
# vim inventory.yaml
Enter fullscreen mode Exit fullscreen mode

Fill the file like this one:

---
- role: Manager
  availability: drain
  hostname: manager-1
  ip: 192.168.100.10
  image: centos/8
  ram: 2048
  cpus: 1
- role: Manager
  availability: drain
  hostname: manager-2
  ip: 192.168.100.11
  image: centos/8
  ram: 2048
  cpus: 1
- role: Manager
  availability: drain
  hostname: manager-3
  ip: 192.168.100.12
  image: centos/8
  ram: 2048
  cpus: 1
- role: Worker
  availability: active
  hostname: worker-1
  ip: 192.168.100.13
  image: centos/8
  ram: 4096
  cpus: 1
- role: Worker
  availability: active
  hostname: worker-2
  ip: 192.168.100.14
  image: centos/8
  ram: 4096
  cpus: 1
- role: Worker
  availability: active
  hostname: worker-3
  ip: 192.168.100.15
  image: centos/8
  ram: 4096
  cpus: 1
Enter fullscreen mode Exit fullscreen mode

This is inventory file of your docker swarm cluster with relevant virtual machine (VM) characteristics.

Vagrant file

Create a Vagrantfile in your current directory, and open the file in you favorite editor:

$ touch Vagrantfile
# code .
Enter fullscreen mode Exit fullscreen mode

Now, it's time to code Vagrantfile using Ruby language...

On top of the file add following code 

# -*- mode: ruby -*-
# vi: set ft=ruby :

require 'yaml'

current_dir = File.dirname(File.expand_path(__FILE__))
servers = YAML.load_file("#{current_dir}/inventory.yaml")
leader = servers.find { |favor| favor['hostname'] == 'manager-1' }

group = "Docker Swarm"
Enter fullscreen mode Exit fullscreen mode

Here, we load the inventory file, and then we define a leader node in the cluster. This node is used to initialize docker swarm.
Also we defined Docker Swarm as a group of VMs in VirtualBox GUI.

Add Vagrant configuration below 

Vagrant.configure("2") do |config|

end
Enter fullscreen mode Exit fullscreen mode

The "2" in Vagrant.configure configures the configuration version.
Inside this block is the place where the main code will be placed.

Define all VMs in accordance with inventory file 

  servers.each do |machine|
    config.vm.define machine['hostname'] do |node|
        node.vm.box = machine['image']
        node.vm.hostname = machine['hostname']
        node.vm.network :private_network, ip: machine['ip']
        node.vm.provider :virtualbox do |vb|
          vb.customize ["modifyvm", :id, "--groups", "/#{group}/#{machine['role']}"]
          vb.name = machine['hostname']
          vb.memory = machine['ram']
          vb.cpus = machine['cpus']
        end
    end
  end
Enter fullscreen mode Exit fullscreen mode

Define hosts file on all VMs 

  servers.each do |machine|
    config.vm.provision "Setup hosts", type: :shell, :args => [machine['ip'], machine['hostname']], inline: <<-SHELL
      sudo echo "$1  $2"  >> /etc/hosts
    SHELL
  end
Enter fullscreen mode Exit fullscreen mode

Setup Linux package repositories 

  config.vm.provision "Setup repositories", type: :shell, privileged: true, inline: <<-SHELL
    sed -i s/mirror.centos.org/vault.centos.org/g /etc/yum.repos.d/CentOS-*.repo
    sed -i s/^mirrorlist=http/#mirrorlist=http/g /etc/yum.repos.d/CentOS-*.repo
    sed -i s/^#.*baseurl=http/baseurl=http/g /etc/yum.repos.d/CentOS-*.repo
    yum clean all
  SHELL
Enter fullscreen mode Exit fullscreen mode

Install Docker Engine on all VMs 

config.vm.provision "Install Docker", type: :docker
Enter fullscreen mode Exit fullscreen mode

Install Redis as a cache in docker container 

  config.vm.provision "Run Redis cache", type: :docker do |d|
    d.run "redis",
      cmd: "--bind 0.0.0.0",
      args: "-p 0.0.0.0:6379:6379"
  end
Enter fullscreen mode Exit fullscreen mode

In our case, Redis cache is used to temporary store docker swarm join tokens.

Initialize docker swarm 

  config.vm.define "manager-1" do |node|
    node.vm.provision "Swarm init", type: :shell, privileged: true, :args => [leader['ip'], leader['availability']],
      inline: "docker swarm init --advertise-addr $1 --availability $2 || true"

    node.vm.provision "Save join-tokens", type: :shell, privileged: true,
      inline: <<-SHELL
        docker exec redis redis-cli set manager-join-token $(docker swarm join-token manager -q)
        docker exec redis redis-cli set worker-join-token $(docker swarm join-token worker -q)
      SHELL
  end
Enter fullscreen mode Exit fullscreen mode

As you can see above, join tokens are saved into Redis cache.

And finally join all VMs to the docker swarm cluster 

  servers.select { |favor| favor['hostname'] != 'manager-1' }.each do |machine|
    config.vm.define machine['hostname'] do |node|
      node.vm.provision "Add a #{machine['role']} to the swarm", type: :shell, privileged: true, 
        :args => [leader['ip'], machine['role'].downcase, machine['availability']],
        inline: <<-SHELL
          token=$(docker exec redis redis-cli -h $1 get $2-join-token)
          docker swarm join --availability $3 --token $token $1 || true
        SHELL
    end
  end
Enter fullscreen mode Exit fullscreen mode

Boot an environment

Bring up a virtual machine

Run the following from your terminal:

$ vagrant up
Enter fullscreen mode Exit fullscreen mode

This command will finish and you will have VMs running with docker swarm.

Check swarm nodes

Connect to a master node using SSH:

$ vagrant ssh manager-1
[vagrant@manager-1 ~]$
Enter fullscreen mode Exit fullscreen mode

Finally, verify the status of the cluster nodes using the following command:

[vagrant@manager-1 ~]$ docker node ls
ID                            HOSTNAME    STATUS    AVAILABILITY   MANAGER STATUS   ENGINE VERSION
illxi93is8446ghgl86lihg1f *   manager-1   Ready     Drain          Leader           26.1.3
es0ktcmq5d9403p16b688knat     manager-2   Ready     Drain          Reachable        26.1.3
j1wnwxnysm763eex9r024h8if     manager-3   Ready     Drain          Reachable        26.1.3
comemxromzh4y8a6ciou6smk8     worker-1    Ready     Active                          26.1.3
smz5sf6vdbzszc850zxwddfie     worker-2    Ready     Active                          26.1.3
p7r1khx4h8famoi9fb0itx8kg     worker-3    Ready     Active                          26.1.3
Enter fullscreen mode Exit fullscreen mode

VirualBox GUI

Now VirualBox GUI should looks like this:

VirualBox GUI

Deploy a service

In order to verify that everything is OK, we will deploy the simplest docker service to a swarm:

$ docker service create --name my_web \
                        --replicas 3 \
                        --publish published=8080,target=80 \
                        nginx:latest
Enter fullscreen mode Exit fullscreen mode

The service is scheduled on available nodes. To confirm that the service was created and started successfully, use the docker service ls command:

[vagrant@manager-1 ~]$ docker service create --name my_web \
>                         --replicas 3 \
>                         --publish published=8080,target=80 \
>                         nginx:latest
s3vgnoaca0p14jgd27ag52th4
overall progress: 3 out of 3 tasks
1/3: running   [==================================================>]
2/3: running   [==================================================>]
3/3: running   [==================================================>]
verify: Service s3vgnoaca0p14jgd27ag52th4 converged
[vagrant@manager-1 ~]$ docker service ls
ID             NAME      MODE         REPLICAS   IMAGE          PORTS
s3vgnoaca0p1   my_web    replicated   3/3        nginx:latest   *:8080->80/tcp
Enter fullscreen mode Exit fullscreen mode

Now, you may open just deployed nginx service in web browser using one of the VM's IP:

Nginx Welcome page

Cleanup

To delete docker service, use the docker service rm command:

[vagrant@manager-1 ~]$ docker service rm my_web
my_web
[vagrant@manager-1 ~]$ docker service ls
ID        NAME      MODE      REPLICAS   IMAGE     PORTS
Enter fullscreen mode Exit fullscreen mode

To stop VMs that Vagrant is managing and remove all the resources created during the machine-creation process, use the vagrant destroy command:

$ vagrant destroy
    worker-3: Are you sure you want to destroy the 'worker-3' VM? [y/N]
Enter fullscreen mode Exit fullscreen mode

To gracefully shut down the VMs, use the vagrant halt command:

$ vagrant halt
==> worker-3: Attempting graceful shutdown of VM...
==> worker-2: Attempting graceful shutdown of VM...
==> worker-2: Forcing shutdown of VM...
==> worker-1: Attempting graceful shutdown of VM...
==> manager-3: Attempting graceful shutdown of VM...
==> manager-2: Attempting graceful shutdown of VM...
==> manager-1: Attempting graceful shutdown of VM...
Enter fullscreen mode Exit fullscreen mode

Summary

In this post, we’ve seen:

  • how to easily setup high-availability (HA) Docker Swarm cluster using Vagrant
  • how to deploy docker service to a swarm cluster
  • how to clean up environment.

The codebase for this article can be found here.

Top comments (0)

Read next

yukaty profile image

Part 2: Implementing Vector Search with OpenAI

yukaty -

githaiga22 profile image

From Zero to Hero: Dockerizing My Go App and Hosting It on Render like a Pro 🚀

Allan Githaiga -

marcos_vilela profile image

[pt-BR] Como utilizei o Vagrant e o Virtual Box para desenvolvimento local

Marcos Vilela -

patfinder profile image

Simple steps to debug docker build

Le Vuong -