Introduction

I made a mistake in my previous posts which I want to correct now. I wrote about Ansible installed in Python virtual environments and as always I wanted to share a code that would work for everyone the same way. For that reason I saved the exact Ansible version (8.0.0) in the requirements.txt. Using specific versions is usually a good idea. That's why we are using specific tags when we are referring to Docker images so every time we start the container we have the same version working the same way.

In case of Ansible without a container the environment can be different for everyone. Most importantly the Python version can be different and the new versions of Ansible might not support older Python versions. No, I'm not going to run Ansible in a container, even though it would be possible and sometimes I do. The reason I don't want to do it now is that I would like you to learn about Ansible and optionally learn about Docker later. Forcing you to install Docker at this point seems to be a bad idea especially if you are not on Linux, because then you would most likely install Docker Desktop, which would mean that you would run a virtual machine just to run an Ansible command and then the simplicity of Ansible is gone.

Table of contents

• Before you begin
  • Requirements
  • Download the already written code of the previous episode
  • Have an inventory file
• Install the Nix package manager
• Using the Nix shell
  • Install Python 3.11
  • Install the pip Python module
  • Install the virtualenv Python module
• Install Python requirements in a virtual environment
• Write a Nix shell script to create the virtual environment
• Conclusion

Before you begin

Requirements

» Back to table of contents «

• You will also need an Ubuntu remote server. I recommend an Ubuntu 22.04 virtual machine.

Download the already written code of the previous episode

» Back to table of contents «

If you started the tutorial with this episode, clone the project from GitHub:

git clone https://github.com/rimelek/homelab.git
cd homelab

If you cloned the project now, or you want to make sure you are using the exact same code I did, switch to the previous episode in a new branch

git checkout -b tutorial.episode.3b tutorial.episode.3

Have the inventory file

» Back to table of contents «

Copy the inventory template

cp inventory-example.yml inventory.yml

And change ansible_host to the IP address of your Ubuntu server that you use for this tutorial, and change ansible_user to the username on the remote server that Ansible can use to log in. If you still don't have an SSH private key, read the Generate an SSH key part of Ansible playbook and SSH keys

Install the Nix package manager

» Back to table of contents «

We don't want to use containers. So what other options do we have? We have to install a specific python version on the host.

1. You could download the source code and build the Python interpreter on your machine. That could work, but building from source code is probably not the way you prefer.
2. You can find a platform-dependent repository, configure it on your machine (different on Mac and Linux) and install it from a repository.
3. Find a way that works on each platform, so you can use the same command everywhere. At least almost everywhere.

Let's choose the third option and find a multiplatform package manager. We could use Homebrew on macOS and Linux, but it is not supported on arm CPUs on Linux. Fortunately we can use Nix to install the latest versions of Python on Mac and also on Linux with arm processors. We don't need the operating system, only the package manager. On windows, you will need Windows Subsystem for Linux (WSL2) to run nix. Yes, that means a kind of virtual machine, but Ansible command doesn't run natively on Windows, so you would need to have a Linux environment anyway.

According to the current installation instructions, the following command installs Nix on macOS:

sh <(curl -L https://nixos.org/nix/install)

and the following command on Linux:

sh <(curl -L https://nixos.org/nix/install) --daemon

If you want to run it on Linux without a daemon that requires root access, use --no-daemon instead of --deamon. I ran it as a daemon as this is the recommended way.

Nix is much more than we will talk about in this post, since we just need a very simple Nix shell.

Using the Nix shell

Install Python 3.11

» Back to table of contents «

We have nix installed, let's run a shell that allows us to use Python 3.11.

nix-shell -p python311

So the required packages come after -p. If you want to install Python 3.11, you just need to write python311 without spaces or dots in the version number. You can go to https://search.nixos.org/packages to search for packages. If Nix is installed properly on your machine, you had to get a new prompt like this:

[nix-shell:~]$

You can check the Python version (python --version) to make sure you have the right version and not the one that is installed outside the Nix shell. Of course, you could have the same version outside the shell so run

which python

to see where the command actually is:

/nix/store/q5labwkn51npa2hp3ibvvhr0xgcfry0q-python3-3.11.4/bin/python

Install the pip Python module

» Back to table of contents «

To install the required Python packages, we need to use pip which is not installed yet. If you had Python installed on your machine, you might have the pip command but that would not use the Python we have just installed in the Nix shell. To use the correct pip command you need to refer to it as a Python module:

python -m pip --version

which will give you an error message like this:

/nix/store/q5labwkn51npa2hp3ibvvhr0xgcfry0q-python3-3.11.4/bin/python: No module named pip

Let's exit the shell and run the following command:

nix-shell -p python311 -p python311Packages.pip

python -m pip --version will now give you a version number:

pip 23.0.1 from /nix/store/x9mkg84c1y97vr434x32g3f7753mcl0l-python3.11-pip-23.0.1/lib/python3.11/site-packages/pip (python 3.11)

Install the virtualenv Python module

» Back to table of contents «

If you try to install the requirements now, it won't work, because you can't write the folders where the Python packages would be stored. That's no problem, since we wanted to create a virtual environment anyway. Again, if you just try to run a virtualenv command that wouldn't be executed by the Python installed in the Nix shell, so we need to install it as a Python module the same way we installed pip. Exit the shell, and run the following command:

nix-shell \
  -p python311 \
  -p python311Packages.pip \
  -p python311Packages.virtualenv

Install Python requirements in a virtual environment

» Back to table of contents «

Now you can create the environment:

python -m virtualenv venv

You could continue with activating the Python environment here in the Nix shell, but you don't actually need the shell anymore unless you want to install other packages like jq (which is often useful by the way) in the shell instead of installing without Nix. I might add more dependencies later, but now it's enough, and you can exit and activate the environment outside the nix shell.

source venv/bin/activate

or you can use the helper scripts I gave you before, mentioned in The first Ansible role and what you can find in my Homelab repository.

It's time to install the requirements:

pip install -r requirements.txt

Notice I used the pip command directly from the terminal since it is available after activating the Python environment.

Write a Nix shell script to create the virtual environment

» Back to table of contents «

Since we used the Nix shell only for creating the Python environment, we can create a script that installs the required nix packages, create the Python environment without interactively activating the Nix shell. Let's call this script "create-nix-env.sh".

#!/usr/bin/env nix-shell
#! nix-shell -i bash
#! nix-shell -p python311
#! nix-shell -p python311Packages.pip
#! nix-shell -p python311Packages.virtualenv
#! nix-shell -I https://github.com/NixOS/nixpkgs/archive/refs/tags/23.05.tar.gz

set -eu -o pipefail

if (( $# == 0 )); then
  >&2 echo "Set the name of the virtual Python environment as the first argument."
  exit 1
fi

env_name="$1"

if [[ ! -d "$env_name" ]]; then
  python3.11 -m virtualenv "$env_name"
fi

source "$env_name/bin/activate"

python3.11 -m pip install -r requirements.txt

Let's break down the script to understand each part.

The first line

#!/usr/bin/env nix-shell

is the shebang line that tells your macOS or Linux that you want to use nix-shell to execute the script. The rest of the comments are interpreted by Nix.

#! nix-shell -i bash

This line will tell Nix that you want to use bash to execute the rest of the file after the comments. I'm sure you recognize the following lines.

#! nix-shell -p python311
#! nix-shell -p python311Packages.pip
#! nix-shell -p python311Packages.virtualenv

Yes, this is how we define the dependencies. The following line defines a specific nix package repository version.

#! nix-shell -I https://github.com/NixOS/nixpkgs/archive/refs/tags/23.05.tar.gz

And the rest of the file is just a normal bash script to create the environment and install the Python requirements. After that you will get your original prompt back and can activate the Python environment to use Ansible or whatever you defined in the requirements.txt.

You could ask why I didn't install Ansible the same way I installed pip or virtualenv. I simply didn't want to rely on nix too much since I have my normal ways to install packages, I have a requirements.txt and I don't really need the nix shell for now, only Python 3.11. This way you can run the following command to create the environment:

./create-nix-env.sh venv

or

./create-nix-env.sh venv-linux

and follow the rest of the tutorial without Nix.

Conclusion

» Back to table of contents «

Often we have many options to solve a problem, and sometimes it's hard to choose. Using the same tool for everything could seem wrong when there is another tool which is designed exactly for what you want or just seems better. On the other hand if you use too many tools the maintenance could be a nightmare. In this case I choose Nix instead of containers, but ultimately it's up to you. If you are familiar with containers, you can run Python in a container even if it means you need Docker Desktop. However, if you want to choose the simplest way, I recommend installing a software from an official repository supported by your operating system. If you want to share your software, keep in mind your audience and what will be the best for them. You can also support alternative solutions if you can maintain it. To be honest, I don't think I would have used Nix if I didn't want share this project since I usually use the latest stable Ubuntu LTS and the latest macOS at least until my machines support it.

What's your opinion? What do you use Nix for?

The final source code of this episode can be found on GitHub:

https://github.com/rimelek/homelab/tree/tutorial.episode.4