DEV Community

loading...
Cover image for Parallel Provisioning with Vagrant and Ansible

Parallel Provisioning with Vagrant and Ansible

Project-42
Working with and for Oracle since 2012 Always searching for new things to learn and new questions to ask Any opinion or mistake are my own :)
・9 min read

Ok, so after installing Vagrant + Libvirt lets see if we can deploy some VMs in parallel adding also some Ansible magic.

For this test, will be using Oracle Linux Boxes.

The first step is to get the Oracle Linux Boxes and do a simple test to make sure the VM creation is working fine.

I just followed similar steps from here (good guide as well to get Vagrant + Libvirt running) but needed to change the URL for the OL8 Boxes, which currently is "https://oracle.github.io/vagrant-projects/boxes/oraclelinux".
More info in Oracle Yum Website

|=| server in ~ ○ → mkdir Vagrant_Oracle

|=| server in ~ ○ → cd Vagrant_Oracle/

|=| server in ~/Vagrant_Oracle ○ → vagrant init oraclelinux/8 https://oracle.github.io/vagrant-projects/boxes/oraclelinux/8.json
A `Vagrantfile` has been placed in this directory. You are now
ready to `vagrant up` your first virtual environment! Please read
the comments in the Vagrantfile as well as documentation on
`vagrantup.com` for more information on using Vagrant.

Enter fullscreen mode Exit fullscreen mode

The Vagrantfile generated is the most simple one, but should be enough to test simple provision

|=| server in ~/Vagrant_Oracle ○ → grep '^[[:blank:]]*[^[:blank:]#;]' Vagrantfile
Vagrant.configure("2") do |config|
  config.vm.box = "oraclelinux/8"
  config.vm.box_url = "https://oracle.github.io/vagrant-projects/boxes/oraclelinux/8.json"
end

|=| server in ~/Vagrant_Oracle ○ →
Enter fullscreen mode Exit fullscreen mode

Let's make sure a simple OL8 Box is created:

|=| server in ~/Vagrant_Oracle ○ →  vagrant up --provider=libvirt
Bringing machine 'default' up with 'libvirt' provider...
==> default: Box 'oraclelinux/8' could not be found. Attempting to find and install...
    default: Box Provider: libvirt
    default: Box Version: >= 0
==> default: Loading metadata for box 'https://oracle.github.io/vagrant-projects/boxes/oraclelinux/8.json'
    default: URL: https://oracle.github.io/vagrant-projects/boxes/oraclelinux/8.json
==> default: Adding box 'oraclelinux/8' (v8.3.183) for provider: libvirt
    default: Downloading: https://yum.oracle.com/boxes/oraclelinux/ol8/OL8U3_x86_64-vagrant-libvirt-b183.box
Progress: 4% (Rate: 8253k/s, Estimated time remaining: 0:01:06)

[.....]

==> default: Uploading base box image as volume into Libvirt storage...
==> default: Creating image (snapshot of base box volume).
==> default: Creating domain with the following settings...
==> default:  -- Name:              Vagrant_Oracle_default
==> default:  -- Domain type:       kvm
==> default:  -- Cpus:              2
==> default:  -- Feature:           apic
==> default:  -- Feature:           acpi
==> default:  -- Memory:            2048M
==> default:  -- Management MAC:
==> default:  -- Loader:
==> default:  -- Nvram:
==> default:  -- Base box:          oraclelinux/8
==> default:  -- Storage pool:      default
==> default:  -- Image:             /var/lib/libvirt/images/Vagrant_Oracle_default.img (37G)
==> default:  -- Volume Cache:      default
==> default:  -- Kernel:
==> default:  -- Initrd:
==> default:  -- Graphics Type:     vnc
==> default:  -- Graphics Port:     -1
==> default:  -- Graphics IP:       127.0.0.1
==> default:  -- Graphics Password: Not defined
==> default:  -- Video Type:        cirrus
==> default:  -- Video VRAM:        16384
==> default:  -- Sound Type:
==> default:  -- Keymap:            en-us
==> default:  -- TPM Path:
==> default:  -- INPUT:             type=mouse, bus=ps2
==> default: Creating shared folders metadata...
==> default: Starting domain.
==> default: Waiting for domain to get an IP address...
==> default: Waiting for SSH to become available...
    default:
    default: Vagrant insecure key detected. Vagrant will automatically replace
    default: this with a newly generated keypair for better security.
    default:
    default: Inserting generated public key within guest...
    default: Removing insecure key from the guest if it's present...
    default: Key inserted! Disconnecting and reconnecting using new SSH key...
==> default: Exporting NFS shared folders...
==> default: Preparing to edit /etc/exports. Administrator privileges will be required...
==> default: Mounting NFS shared folders...

|=| server in ~/Vagrant_Oracle ○ →
Enter fullscreen mode Exit fullscreen mode

Since I have the intention to create multiple Boxes, decided to create a new libvirt storage pool for Vagrant and avoid any space issues

|=| server in /home/kvm/HDD/Vagrant ○ → virsh pool-define-as --name Vagrant --type dir --target /home/kvm/HDD/Vagrant
Pool Vagrant defined

|=| server in /home/kvm/HDD/Vagrant ○ → virsh pool-start Vagrant
Pool Vagrant started

|=| server in /home/kvm/HDD/Vagrant ○ → virsh pool-autostart Vagrant
Pool Vagrant marked as autostarted

|=| server in /home/kvm/HDD/Vagrant ○ → virsh pool-list --all
 Name       State    Autostart
--------------------------------
 default    active   yes
 disks      active   yes
 kvm        active   yes
 solifugo   active   yes
 Vagrant    active   yes


|=| server in /home/kvm/HDD/Vagrant ○ → df -h .
Filesystem      Size  Used Avail Use% Mounted on
/dev/sdc1       1.8T  960G  780G  56% /home/kvm/HDD

|=| server in /home/kvm/HDD/Vagrant ○ →
Enter fullscreen mode Exit fullscreen mode

Let's start now changing the Vagrantfile to create multiple VMs.
If you see something familar, is because I just used a generic one from Vagrant Website: Ansible Provisioner
This is our Vagrantfile (added some comments for each option)

|=| server in ~/Vagrant_Oracle ○ → cat Vagrantfile
# -*- mode: ruby -*-
# vi: set ft=ruby :

Vagrant.configure("2") do |config|
  config.vm.synced_folder ".", "/vagrant", :disabled => true # Disable NFS mount
  config.vm.provider :libvirt do |libvirt| # Tells vagrant to use libvirt
    libvirt.storage_pool_name = "Vagrant" # Use this Storage pool
    libvirt.default_prefix = "Vagrant_" # Shows Vagrant_ prefix in Virsh status
  end
  config.vm.box = "oraclelinux/8" # Box Dsitribution/version
  config.vm.box_url = "https://oracle.github.io/vagrant-projects/boxes/oraclelinux/8.json" # Place to locate the Box Image
  N = 3 # Number of VMs we want to deploy
  (1..N).each do |machine_id|
    config.vm.define "dbnode#{machine_id}" do |machine| #VM name
      machine.vm.hostname = "dbnode#{machine_id}" #VM hostname
      # Only execute once the Ansible provisioner,
      # when all the machines are up and ready.
      if machine_id == N
        machine.vm.provision :ansible do |ansible| #Vagrant to call Ansible
          # Disable default limit to connect to all the machines
          ansible.limit = "all" # Make sure Ansible check all VMs
          ansible.playbook = "Ansible_basic_playbook.yml" # Ansible playbook to execute
        end
      end
    end
  end

|=| server in ~/Vagrant_Oracle ○ →
Enter fullscreen mode Exit fullscreen mode

Also, we will execute a simple Ansible Task to just create the user oracle and some user groups as well, just to see if that works in all 3 VMs we are creating, nothing really difficult (in theory)

|=| server in ~/Vagrant_Oracle ○ → cat Ansible_basic_playbook.yml
---
- name: Playbook
  hosts: all
  become: yes
  vars:
    users:
    -  "oracle"

  tasks:

  - name: Create group
    group:
      name: "{{ item }}"
      state: present
    with_items:
      - "admin"
      - "oinstall"
      - "oracle"

  - name: "Create user accounts"
    user:
      name: "{{ item }}"
      password: "{{ 'Welcome1' | password_hash('sha512') }}"
      groups: "oracle,oinstall,admin"
    with_items: "{{ users }}"

  - name: "Allow admin users to sudo without a password"
    lineinfile:
      dest: "/etc/sudoers"
      state: "present"
      regexp: "^%admin"
      line: "%admin ALL=(ALL) NOPASSWD: ALL"

|=| server in ~/Vagrant_Oracle ○ →
Enter fullscreen mode Exit fullscreen mode

Let's try it out:

|=| server in ~/Vagrant_Oracle ○ → vagrant up
Bringing machine 'dbnode1' up with 'libvirt' provider...
Bringing machine 'dbnode2' up with 'libvirt' provider...
Bringing machine 'dbnode3' up with 'libvirt' provider...
==> dbnode2: Checking if box 'oraclelinux/8' version '8.3.183' is up to date...
==> dbnode1: Checking if box 'oraclelinux/8' version '8.3.183' is up to date...
==> dbnode3: Checking if box 'oraclelinux/8' version '8.3.183' is up to date...
==> dbnode2: Creating image (snapshot of base box volume).
==> dbnode3: Creating image (snapshot of base box volume).
==> dbnode1: Creating image (snapshot of base box volume).
==> dbnode3: Creating domain with the following settings...
==> dbnode2: Creating domain with the following settings...
==> dbnode1: Creating domain with the following settings...
==> dbnode1:  -- Name:              Vagrant_dbnode1
==> dbnode2:  -- Name:              Vagrant_dbnode2
==> dbnode3:  -- Name:              Vagrant_dbnode3
==> dbnode2:  -- Domain type:       kvm
==> dbnode3:  -- Domain type:       kvm
==> dbnode1:  -- Domain type:       kvm
==> dbnode1:  -- Cpus:              2
==> dbnode1:  -- Feature:           apic
==> dbnode1:  -- Feature:           acpi
==> dbnode1:  -- Memory:            2048M
==> dbnode2:  -- Cpus:              2
==> dbnode3:  -- Cpus:              2
==> dbnode3:  -- Feature:           apic
==> dbnode2:  -- Feature:           apic
==> dbnode1:  -- Management MAC:
==> dbnode1:  -- Loader:
==> dbnode3:  -- Feature:           acpi
==> dbnode1:  -- Nvram:
==> dbnode2:  -- Feature:           acpi
==> dbnode1:  -- Base box:          oraclelinux/8
==> dbnode1:  -- Storage pool:      Vagrant
==> dbnode1:  -- Image:             /home/kvm/HDD/Vagrant/Vagrant_dbnode1.img (37G)
==> dbnode2:  -- Memory:            2048M
==> dbnode1:  -- Volume Cache:      default
==> dbnode3:  -- Memory:            2048M
==> dbnode1:  -- Kernel:
==> dbnode3:  -- Management MAC:
==> dbnode2:  -- Management MAC:
==> dbnode1:  -- Initrd:
==> dbnode3:  -- Loader:
==> dbnode1:  -- Graphics Type:     vnc
==> dbnode3:  -- Nvram:
==> dbnode1:  -- Graphics Port:     -1
==> dbnode3:  -- Base box:          oraclelinux/8
==> dbnode2:  -- Loader:
==> dbnode3:  -- Storage pool:      Vagrant
==> dbnode1:  -- Graphics IP:       127.0.0.1
==> dbnode3:  -- Image:             /home/kvm/HDD/Vagrant/Vagrant_dbnode3.img (37G)
==> dbnode2:  -- Nvram:
==> dbnode3:  -- Volume Cache:      default
==> dbnode1:  -- Graphics Password: Not defined
==> dbnode2:  -- Base box:          oraclelinux/8
==> dbnode3:  -- Kernel:
==> dbnode1:  -- Video Type:        cirrus
==> dbnode2:  -- Storage pool:      Vagrant
==> dbnode3:  -- Initrd:
==> dbnode2:  -- Image:             /home/kvm/HDD/Vagrant/Vagrant_dbnode2.img (37G)
==> dbnode3:  -- Graphics Type:     vnc
==> dbnode1:  -- Video VRAM:        16384
==> dbnode1:  -- Sound Type:
==> dbnode2:  -- Volume Cache:      default
==> dbnode1:  -- Keymap:            en-us
==> dbnode3:  -- Graphics Port:     -1
==> dbnode1:  -- TPM Path:
==> dbnode1:  -- INPUT:             type=mouse, bus=ps2
==> dbnode3:  -- Graphics IP:       127.0.0.1
==> dbnode2:  -- Kernel:
==> dbnode3:  -- Graphics Password: Not defined
==> dbnode3:  -- Video Type:        cirrus
==> dbnode3:  -- Video VRAM:        16384
==> dbnode2:  -- Initrd:
==> dbnode3:  -- Sound Type:
==> dbnode3:  -- Keymap:            en-us
==> dbnode2:  -- Graphics Type:     vnc
==> dbnode2:  -- Graphics Port:     -1
==> dbnode3:  -- TPM Path:
==> dbnode2:  -- Graphics IP:       127.0.0.1
==> dbnode3:  -- INPUT:             type=mouse, bus=ps2
==> dbnode2:  -- Graphics Password: Not defined
==> dbnode2:  -- Video Type:        cirrus
==> dbnode2:  -- Video VRAM:        16384
==> dbnode2:  -- Sound Type:
==> dbnode1: Creating shared folders metadata...
==> dbnode2:  -- Keymap:            en-us
==> dbnode1: Starting domain.
==> dbnode2:  -- TPM Path:
==> dbnode1: Waiting for domain to get an IP address...
==> dbnode2:  -- INPUT:             type=mouse, bus=ps2
==> dbnode3: Creating shared folders metadata...
==> dbnode3: Starting domain.
==> dbnode3: Waiting for domain to get an IP address...
==> dbnode2: Creating shared folders metadata...
==> dbnode2: Starting domain.
==> dbnode2: Waiting for domain to get an IP address...
==> dbnode1: Waiting for SSH to become available...
==> dbnode2: Waiting for SSH to become available...
==> dbnode3: Waiting for SSH to become available...
    dbnode1:
    dbnode1: Vagrant insecure key detected. Vagrant will automatically replace
    dbnode1: this with a newly generated keypair for better security.
    dbnode2:
    dbnode2: Vagrant insecure key detected. Vagrant will automatically replace
    dbnode2: this with a newly generated keypair for better security.
    dbnode3:
    dbnode3: Vagrant insecure key detected. Vagrant will automatically replace
    dbnode3: this with a newly generated keypair for better security.
    dbnode2:
    dbnode2: Inserting generated public key within guest...
    dbnode1:
    dbnode1: Inserting generated public key within guest...
    dbnode3:
    dbnode3: Inserting generated public key within guest...
    dbnode2: Removing insecure key from the guest if it's present...
    dbnode1: Removing insecure key from the guest if it's present...
    dbnode3: Removing insecure key from the guest if it's present...
    dbnode2: Key inserted! Disconnecting and reconnecting using new SSH key...
    dbnode1: Key inserted! Disconnecting and reconnecting using new SSH key...
    dbnode3: Key inserted! Disconnecting and reconnecting using new SSH key...
==> dbnode2: Setting hostname...
==> dbnode1: Setting hostname...
==> dbnode3: Setting hostname...
==> dbnode3: Running provisioner: ansible...
    dbnode3: Running ansible-playbook...

PLAY [Playbook] ****************************************************************

TASK [Gathering Facts] *********************************************************
[WARNING]: Platform linux on host dbnode2 is using the discovered Python
interpreter at /usr/libexec/platform-python, but future installation of another
Python interpreter could change this. See https://docs.ansible.com/ansible/2.9/
reference_appendices/interpreter_discovery.html for more information.
ok: [dbnode2]
[WARNING]: Platform linux on host dbnode3 is using the discovered Python
interpreter at /usr/libexec/platform-python, but future installation of another
Python interpreter could change this. See https://docs.ansible.com/ansible/2.9/
reference_appendices/interpreter_discovery.html for more information.
ok: [dbnode3]
[WARNING]: Platform linux on host dbnode1 is using the discovered Python
interpreter at /usr/libexec/platform-python, but future installation of another
Python interpreter could change this. See https://docs.ansible.com/ansible/2.9/
reference_appendices/interpreter_discovery.html for more information.
ok: [dbnode1]

TASK [Create group] ************************************************************
changed: [dbnode3] => (item=admin)
changed: [dbnode1] => (item=admin)
changed: [dbnode2] => (item=admin)
changed: [dbnode3] => (item=oinstall)
changed: [dbnode1] => (item=oinstall)
changed: [dbnode2] => (item=oinstall)
changed: [dbnode3] => (item=oracle)
changed: [dbnode1] => (item=oracle)
changed: [dbnode2] => (item=oracle)

TASK [Create user accounts] ****************************************************
changed: [dbnode2] => (item=oracle)
changed: [dbnode1] => (item=oracle)
changed: [dbnode3] => (item=oracle)

TASK [Allow admin users to sudo without a password] ****************************
changed: [dbnode3]
changed: [dbnode1]
changed: [dbnode2]

PLAY RECAP *********************************************************************
dbnode1                    : ok=4    changed=3    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0
dbnode2                    : ok=4    changed=3    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0
dbnode3                    : ok=4    changed=3    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0


|=| server in ~/Vagrant_Oracle ○ →
Enter fullscreen mode Exit fullscreen mode

and voilà! we have our 3 "dbnodes" running

|=| server in ~/Vagrant_Oracle ○ → vagrant ssh dbnode3

Welcome to Oracle Linux Server release 8.3 (GNU/Linux 5.4.17-2036.100.6.1.el8uek.x86_64)

The Oracle Linux End-User License Agreement can be viewed here:

  * /usr/share/eula/eula.en_US

For additional packages, updates, documentation and community help, see:

  * https://yum.oracle.com/

Last login: Wed Jan  6 14:07:34 2021 from 192.168.121.1
[vagrant@dbnode3 ~]$

|=| server in ~/Vagrant_Oracle ○ → virsh list
 Id   Name              State
---------------------------------
 16   Vagrant_dbnode1   running
 17   Vagrant_dbnode3   running
 18   Vagrant_dbnode2   running


|=| server in ~/Vagrant_Oracle ○ →
Enter fullscreen mode Exit fullscreen mode

If you dont like the fact that you need to be in the Vagrant work folder and execute "vagrant ssh" to connect to your system, you can just add the "vagrant ssh-config" to your user "ssh config" like you can see below:

|=| server in ~/Vagrant_Oracle ○ → vagrant ssh-config >> ~/.ssh/config

|=| server in ~/Vagrant_Oracle ○ → cd

|=| server in ~ ○ → ssh dbnode1

Welcome to Oracle Linux Server release 8.3 (GNU/Linux 5.4.17-2036.100.6.1.el8uek.x86_64)

The Oracle Linux End-User License Agreement can be viewed here:

  * /usr/share/eula/eula.en_US

For additional packages, updates, documentation and community help, see:

  * https://yum.oracle.com/

Last login: Tue Jan  5 15:11:58 2021 from 192.168.121.1
[vagrant@dbnode1 ~]$
Enter fullscreen mode Exit fullscreen mode

By the way, if your VMs are already up but you need to run Ansible playbook again, you can just "vagrant provision"

|=| server in ~/Vagrant_Oracle ○ → vagrant provision
==> dbnode3: Running provisioner: ansible...
    dbnode3: Running ansible-playbook...

PLAY [Playbook] ****************************************************************

TASK [Gathering Facts] *********************************************************
[WARNING]: Platform linux on host dbnode1 is using the discovered Python
interpreter at /usr/libexec/platform-python, but future installation of another
Python interpreter could change the meaning of that path. See https://docs.ansi
ble.com/ansible/2.10/reference_appendices/interpreter_discovery.html for more
information.
ok: [dbnode1]
[WARNING]: Platform linux on host dbnode3 is using the discovered Python
interpreter at /usr/libexec/platform-python, but future installation of another
Python interpreter could change the meaning of that path. See https://docs.ansi
ble.com/ansible/2.10/reference_appendices/interpreter_discovery.html for more
information.
ok: [dbnode3]
[WARNING]: Platform linux on host dbnode2 is using the discovered Python
interpreter at /usr/libexec/platform-python, but future installation of another
Python interpreter could change the meaning of that path. See https://docs.ansi
ble.com/ansible/2.10/reference_appendices/interpreter_discovery.html for more
information.
ok: [dbnode2]

TASK [Create group] ************************************************************
ok: [dbnode3] => (item=admin)
ok: [dbnode1] => (item=admin)
ok: [dbnode2] => (item=admin)
ok: [dbnode3] => (item=oinstall)
ok: [dbnode1] => (item=oinstall)
ok: [dbnode2] => (item=oinstall)
ok: [dbnode3] => (item=oracle)
ok: [dbnode1] => (item=oracle)
ok: [dbnode2] => (item=oracle)

TASK [Create user accounts] ****************************************************
changed: [dbnode1] => (item=oracle)
changed: [dbnode3] => (item=oracle)
changed: [dbnode2] => (item=oracle)

TASK [Allow admin users to sudo without a password] ****************************
ok: [dbnode3]
ok: [dbnode1]
ok: [dbnode2]

PLAY RECAP *********************************************************************
dbnode1                    : ok=4    changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0
dbnode2                    : ok=4    changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0
dbnode3                    : ok=4    changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0


|=| server in ~/Vagrant_Oracle ○ →
Enter fullscreen mode Exit fullscreen mode

Discussion (0)