Introduction

People who know me, also know that I usually talk and write about Docker or other types of containers. I like Docker, and when you like something, sometimes you want to solve everything with it. Then you realize, there are other tools as well, and you need to learn those, because in real life in a real project what you already know is not enough. And when you want to solve everything with a single tool, you will often have a bad headache, because it is just not appropriate for your goals. I think, you should be able to use different tools at least at some level and when you really need to do something as perfectly as it is possible, you can work with specialists. Until then, you may need to build your test environment at home which requires learning about virtual machines, containers, configuring network and find something that can help you constantly create, remove and recreate everything you have, and sometimes with a completely different configuration. Of course, you could do it manually and also make a lot of mistakes.

My goal is to show you a way to build your local environment using cheap devices that you already have. It can be your laptop, your PC and maybe some routers that you can afford. It won't be perfect, and you will reach the point when you don't have enough resources, but at least you can try some concepts until you can decide whether you want to buy a more expensive environment in the cloud or not.

Since I know learning about these things can be hard and make you want to pull your hair out, I want to show you step by step how I have learnt to build my environment which I later found out that it is called a "home lab".

To avoid doing everything manually you can find many tools, but since I use Ansible most of the time, I will show you how Ansible can be used, and hopefully it becomes a series of articles, and you can learn about many things in the meantime.

If you prefer to watch a video about it instead, you can also watch it on YouTube:

Table of contents

• What is Ansible for?
• Install Ansible
• Creating an inventory file
• The first playbook
• Create a file on the remote server using Ansible
• Conclusion

What is Ansible for?

» Back to table of contents «

I assume many of you already know about Docker. If you know about Docker, you most likely know about the Dockerfile which is to describe how the Docker image should be built. Let's see the following example:

FROM ubuntu:22.04

RUN apt-get update \
 && apt-get install --no-install-recommends \
    -y nano 

You can use this Dockerfile to build an image and push the image to a public or even a private registry so other developers can use the same image to run a similar container with the same dependencies. The fact is you don't always need a container. Very often you just want to install or configure something on a physical machine, and you want to repeat it on multiple machines. Now this is what Ansible is for. It is a way to describe an Infrastructure as Code which we also refer to as IaC.

Install Ansible

» Back to table of contents «

Ansible is written in python, so you need python on the so-called Ansible controller (usually your workstation) and also on the remote machine on which you want to run commands.

Note: Technically, python it is not required on remote servers, but without python your options are very limited, so it wouldn't really make sense to use Ansible.

Obviously you need to install python first, which I won't cover in this article. So the next step is to install Ansible, which I don't like to do globally, because some of my projects may require an older version of Ansible while other projects could use newer versions, so I create a virtual python environment. You could do it multiple ways, but let's see one way.

First you need a module that can create a virtual environment. On a Debian based system, you would install it this way:

sudo apt install python3-venv

Then you need to create the environment, which means you will create a folder containing the actual python environment, so let's create a project folder and a virtual environment in it.

mkdir iac
cd iac
python3 -m venv venv

Well, the first venv is the name of the module. The second is the folder that you want to create, but it is usually venv too. Now all you need is to activate the environment.

source venv/bin/activate

Now that python changed your environment variables when you use pip you will use the one installed in the virtual environment. We need pip to install Ansible, but first we need a requirements.txt with the following content:

ansible-core
ansible

ansible in that file would have been enough, since the Ansible core is the dependency of Ansible, however, you can install only the Ansible core and install specific collections from Ansible Galaxy when you need it.

Now install the requirements:

pip install -r requirements.txt

Depending on your network and the platform you are using (Mac vs Linux) it can sometimes take a long time to finish. After that, since you want to be able to repeat it and install the same version which you know works, because you tested it, you need to check the versions of ansible and ansible-core by running

pip freeze

You will see something like this:

ansible==8.0.0
ansible-core==2.15.0
cffi==1.15.1
cryptography==41.0.1
Jinja2==3.1.2
MarkupSafe==2.1.3
packaging==23.1
pycparser==2.21
PyYAML==6.0
resolvelib==1.0.1

You could copy the whole output into the requirements.txt, but it is enough to copy the first two lines only and override the original content with it.

ansible==8.0.0
ansible-core==2.15.0

Creating an inventory file

» Back to table of contents «

Since Ansible is for running commands on remote servers you need to tell Anisble what those servers are and how they can be accessed to. This is what the inventory file is for. A very simple inventory file is like this:

all:
  vars:
    ansible_user: ta
  hosts:
    ta-lxlt:
      ansible_host: 192.168.4.58

You can define not just servers but also a group of servers and different groups can have different parameters. all is a special group magically containing all of your servers even if you don't define this group in the inventory file. Because I don't want to talk about groups yet, I will define my single server (which is a laptop by the way) in the group called all.

As you can see in the example, you can define variables and hosts. The vars section of a group is for variables that must be available for each host when you run the commands on the remote servers. The hosts section will contain the definition of the remote servers and each server becomes a new section in the YAML file (hosts is basically a dictionary). Everything that you define under the section of a host is a variable that can or must be available only for that specific host, for example, because each host will have a different value. In the example it is a different IP address which is unique. The username doesn't have to be unique so if you have multiple servers with the same user in it, you can define the ansible_user variable for the whole group so your inventory file can be shorter. Do it only if it really helps you. If it becomes even harder to read the file, it is fine to define all the variables in the section of the hosts directly.

The first playbook

» Back to table of contents «

A playbook is a YAML file that will "tell" Ansible what you actually want to run on which group of hosts. The following playbook is a very simple, but not a really useful playbook and the name of the file can be playbook.yml.

- name: Play 1
  hosts: all
  tasks:

    - name: Who am I
      ansible.builtin.command: whoami

This file is a list of plays. We didn't talk about plays yet, but all you need to know for now that you can have multiple plays in one playbook, and different plays in one playbook can run tasks on different hosts. Since a playbook contains a list, we had to start the file with a dash character which is an indication of an item in a list.

A play can have a name. It doesn't have to have a name and I usually don't set the name, but using names can be helpful when you check the logs or the output of the ansible commands.

The second parameter is hosts which is actually a name of a group of hosts or a list of groups. Since we have only one group, we can use it as a string. Now we know that the playbook called "Play 1" will run commands on "all" hosts, but we also need to define the commands to run.

I intentionally used the word "command" until know, but in case of Ansible we are talking about tasks. The difference is that Ansible can use a Python module to do its job instead of just running a simple Linux command. One of the modules is "command" and that is what lets you define an actual Linux command to run on the remote. A module is often defined right after the name of the task. It can be strange first, since there is nothing to tell you what is a module name and what is a parameter for something else in the task. In time, you will get used to it, but in recent versions of Ansible you can also use prefixes for module names. This is what we call Fully qualified collection name (FQCN). Yes, I agree, it should be fully qualified module name, but I don't make the rules here :)

If you want to know about a prefix of a specific module, you need to check the documentation.

Run the first playbook:

ansible-playbook -i inventory.yml playbook.yml

The output should be like this:

PLAY [Play 1] **************************************************************************************************************************************************

TASK [Gathering Facts] *****************************************************************************************************************************************
fatal: [ta-lxlt]: UNREACHABLE! => {"changed": false, "msg": "Failed to connect to the host via ssh: ta@192.168.4.58: Permission denied (publickey,password).", "unreachable": true}

PLAY RECAP *****************************************************************************************************************************************************
ta-lxlt                    : ok=0    changed=0    unreachable=1    failed=0    skipped=0    rescued=0    ignored=0

This is expected, because Ansible doesn't know the password of the user, and we didn't define an SSH key either. Using an SSH key is usually recommended, but Ansible can also ask for a password:

ansible-playbook -i inventory.yml playbook.yml --ask-pass

Note: On macOS, it wouldn't work because macOS doesn't support the required dependencies to ask for the password instead of using SSH keys. If you don't have a Linux host, you can also use Multipass to run an Ubuntu virtual machine and mount the project folder into the VM

multipass launch \
  --mount $PWD:/home/ubuntu/projects/iac \
  --cpus 2 \
  --memory 2G \
  --name ansible-controller \
  22.04

multipass shell ansible-controller
cd projects/iac

For more details, please, watch the video mentioned in the Introduction.

On Linux, after typing your password you would get an output like this:

PLAY [Play 1] *************************************************************************************************************************************************************************

TASK [Gathering Facts] ****************************************************************************************************************************************************************
fatal: [ta-lxlt]: FAILED! => {"msg": "to use the 'ssh' connection type with passwords or pkcs11_provider, you must install the sshpass program"}

PLAY RECAP ****************************************************************************************************************************************************************************
ta-lxlt                    : ok=0    changed=0    unreachable=0    failed=1    skipped=0    rescued=0    ignored=0

This is when we need to install "sshpass".

sudo apt install sshpass

If you run the ansible-playbook command again, unless you have already logged in to the remote server using the same IP address and accepted the fingerprint of the remote server, you could get an error message like this:

PLAY [Play 1] *************************************************************************************************************************************************************************

TASK [Gathering Facts] ****************************************************************************************************************************************************************
fatal: [ta-lxlt]: FAILED! => {"msg": "Using a SSH password instead of a key is not possible because Host Key checking is enabled and sshpass does not support this.  Please add this host's fingerprint to your known_hosts file to manage this host."}

PLAY RECAP ****************************************************************************************************************************************************************************
ta-lxlt                    : ok=0    changed=0    unreachable=0    failed=1    skipped=0    rescued=0    ignored=0

The easiest way to solve this is just logging in to the server without ansible or at least trying to log in, and when you get the prompt to type "yes", you do that, and you can press CTRL+C to cancel the login process.

ssh ta@192.168.4.58

Now you run the command again, and Ansible should run the playbook successfully:

PLAY [Play 1] *************************************************************************************************************************************************************************

TASK [Gathering Facts] ****************************************************************************************************************************************************************
ok: [ta-lxlt]

TASK [Who am I] ***********************************************************************************************************************************************************************
changed: [ta-lxlt]

PLAY RECAP ****************************************************************************************************************************************************************************
ta-lxlt                    : ok=2    changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

The only problem is that you don't get the output of the command you ran. It's okay, since you usually don't want to get it. You just want to see if the task changed anything. We just ran the whoami command which doesn't change anything and the command module can't decide it, so it will just always show "changed". If you want to get the output of the command, you can register a variable and save the output to the variable like this:

- name: Play 1
  hosts: all
  tasks:

     - name: Who am I
       ansible.builtin.command: whoami
       register: _command

     - name: debug
       ansible.builtin.debug:
         var: _command

This time I will only show you the output of the debug task:

ok: [ta-lxlt] => {
    "_command": {
        "changed": true,
        "cmd": [
            "whoami"
        ],
        "delta": "0:00:00.001844",
        "end": "2023-06-10 15:20:18.686600",
        "failed": false,
        "msg": "",
        "rc": 0,
        "start": "2023-06-10 15:20:18.684756",
        "stderr": "",
        "stderr_lines": [],
        "stdout": "ta",
        "stdout_lines": [
            "ta"
        ]
    }
}

The debug module expects either the "msg" or the "var" parameter. Since we wanted to see the content of a variable, we needed to use "var". You could change the debug task to look like this:

     - name: debug
       ansible.builtin.debug:
         var: _command.stdout

and get a much shorter output:

TASK [debug] **************************************************************************************************************************************************************************
ok: [ta-lxlt] => {
    "_command.stdout": "ta"
}

Create a file on the remote server using Ansible

» Back to table of contents «

The previous playbook worked, but it didn't change anything. Sometimes it is useful to get more information about a server, but it is more useful to actually change something. The simplest change you can do is to create a file. Use the following content in your playbook.yml

- name: Play 1
  hosts: all
  tasks:

    - name: Create Hello World file
      ansible.builtin.copy:
        content: "Hello World"
        dest: /home/ta/hello-world.txt

This time we use the "copy" module in the ansible.builtin collection. You could copy a file from the host to the remote server or multiple remote servers, but you can also just save the content in the "content" parameter of the "copy" module and define the destination in the parameter called "dest". That is the path of the file on the remote server into which you want to save the content.

It is important to know, that you will override the content on the remote server when you change the "content" parameter in the playbook and run the playbook again. Since we can finally change something, assuming your username is "ta" which is probably not true, the first time you would see something like this:

PLAY [Play 1] *************************************************************************************************************************************************************************

TASK [Gathering Facts] ****************************************************************************************************************************************************************
ok: [ta-lxlt]

TASK [Create Hello World file] ********************************************************************************************************************************************************
changed: [ta-lxlt]

PLAY RECAP ****************************************************************************************************************************************************************************
ta-lxlt                    : ok=2    changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

As you can see, the status of the "Create Hello World file" is "changed". When you run the playbook again, Ansible will check if the file exists with the required content and if it does, it will show "ok".

PLAY [Play 1] *************************************************************************************************************************************************************************

TASK [Gathering Facts] ****************************************************************************************************************************************************************
ok: [ta-lxlt]

TASK [Create Hello World file] ********************************************************************************************************************************************************
ok: [ta-lxlt]

PLAY RECAP ****************************************************************************************************************************************************************************
ta-lxlt                    : ok=2    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

If you want this example to work with any username dynamically, you could use the whoami task and use templating in the copy task:

- name: Play 1
  hosts: all
  tasks:

    - name: Whoami
      ansible.builtin.command: whoami
      register: _command

    - name: Create Hello World file
      ansible.builtin.copy:
        content: "Hello World"
        dest: /home/{{ _command.stdout }}/hello-world.txt

Ansible supports Jinja templates, so variables put between double curly brackets can be used in parameter values. There would be a better way to detect usernames, but we have learnt enough for today.

Conclusion

» Back to table of contents «

It was probably the simplest playbook you could run. It is still not really useful, but this is how you need to start. Understand the basics first, and when you finally understand it, you can do more complicated things. That's what we will do next time.

If you didn't understand everything written in this post, you can watch me running these commands on YouTube: https://youtu.be/K9grKS335Mo

Note: The part about Jinja templates was added later, so you can try the example without crating a user called "ta".

The final source code of this episode can be found on GitHub:

https://github.com/rimelek/homelab/tree/tutorial.episode.1