'Dockerfiles' for LXD

tldr;

freckles can be used as a generic, composable 'Dockerfile' replacement for LXD. To try it out, on a system that has freckles and LXD installed and working, put the following content into a file 'example-lxd.frecklet':

- nginx-service
- nginx-vhost-from-folder:
    document_root: /var/www/html
    default_server: true
    use_https: false
- file-with-content:
    owner: www-data
    path: /var/www/html/index.html
    content: |
      <h1><i>freckles</i> says "Hello World", from {{:: from ::}}!</h1>

This will act as our 'Dockerfile' and describes how to install and configure nginx to serve a static webpage. To create an LXD image from it, execute:

$ frecklecute lxd-image-from-frecklet-file \
     --install-packer \
     --source-image images:debian/10 \
     --image-name freckles-image \
     --frecklet-path $(pwd)/example-lxd.frecklet \
     --frecklet-vars '{"from": "my first image"}'

Once that has finished, create a container from the new 'freckles-image' image:

$ lxc launch freckles-image freckles-test

To test whether the Nginx service that runs in the container works, issue:

$ curl http://<container_ip>
<h1><i>freckles</i> says "Hello World", from my first image!</h1>

Long-ish preamble

I think Dockerfiles played a fairly big part in Dockers success and adoption early on. They are simple to read/understand, flexible enough to be useful beyond basic tasks, and composable (if you squint your eyes a bit and look at 'inheriting from another Docker image' as composition).

Of course, there are other factors for Dockers success, being at the right time at the right place (a.k.a. luck) among them, but Dockerfiles are certainly important. Docker containers are usually considered to be 'application containers', and part of a microservice-type architecture. In practices, that is (and probably never was) 100% true, there are a lot of people trying to force Docker into something else, for example coming up with 'init-systems' for Docker, trying to run webapps incl. Database and Memcache and whatever else in a single container. Think of that what you will, it's still a testament for Dockers success, and the flexibility of Dockerfiles, because they actually allow for all this to be built.

Now, if we just accept the premise there are cases where its advantageous to have multiple services on the same host without the overhead of a virtual network connecting all of them (and we could certainly argue when that makes and doesn't make sense), we'll sooner or later stumble upon LXD (I'll ignore the more low-level LXC, but I acknowledge that sometimes it'd be a better fit).

LXD is a so-called 'system container manager' (as opposed to 'application container'), which means it 'contains' whole operating systems, including a 'normal' init system. An LXD container behaves like a physical or virtual machine, except for some permission-related things, but without much overhead (like 'real' VMs, for example).

LXD does not have an equivalent to a Dockerfile though (ignoring 'cloud-init' for now -- using freckles with 'cloud-init- is a topic for a whole other blog post). One reason for this is that it's not really necessary. Since an LXD container behaves like a normal physical or virtual machine, we can use the same tooling to install applications, and generally get a container into the state we want it to be. And for the case where sysadmin/devops-professionals do the state-making, that is fine, they know what to do, and how to do it in an automated way. Using tools like Ansible, Puppet. Or maybe Packer, Terraform, etc.

The one advantage a 'Dockerfile for LXD' would have in that situation is to bring easy reproducability, re-usability, and just general 'orderlyness' to LXD for people who don't have to deal with those things usually. In the same way Dockerfiles brought those things along with Docker. All you need is a single text file, and you'll be able to replicate and costumize a Docker image. I think we can't overestimate the effect of a technology that allows people who know what they are doing (those who create Dockerfiles) to share what they are doing (Dockerfiles) with people who wouldn't have known how to do that particular thing (at least without spending a few hours on Stackoverflow) or can't be bothered, and who now can just re-use that particular thing.

A thing which, by the way, now only has to be created once, and which then can be continuously improved upon by a community of people (who ideally know what they are doing). It's like using a specialized library in any programming language, it allows you to focus on your core problem as an application developer, without too much distraction on those tangential details you'd have to worry about otherwise. It's really not that deep of a problem, everything considered. But it's wide-spread, and prevalent to the point that we often don't even see and recognize it in our daily (working) life, which means we are not taking advantage of the already existing solutions for such a problem.

This is, incidentally, exactly what freckles tries to solve too, just in a more generic way than a Dockerfile does. Allow for re-usable recipes to bring computational environments into a certain state, technolgy- and stack-independent.

Which, finally, brings me to the topic of this post: using frecklets as sort of re-usable 'Dockerfile-replacements'. In the context of LXD. Of course, since they are re-usable, you can use the same frecklet you use in LXD for Docker too (with some caveats), or, to provision a remote server (as I'll show at the end of this post).

Using freckles with LXD

There are two separate areas where we can choose to use freckles in combination with LXD:

• describing the content/state of the images
• building the actual LXD images

In addition, we can use freckles to install and configure LXD:

Optional: using freckles to install LXD

Note: the following does not setup internet connectivity for containers who are connected to the newly created bridge 'lxbr0'. This has to be done manually (for now), e.g. by doing iptables -t nat -A POSTROUTING -s <bridge_subnet>.0/24 -o eth0 -j MASQUERADE. This will be done automatically sometime in the future, hopefully.

For this, we can use the lxd-service frecklet:

$ frecklecute lxd-service --user markus

Under the hood, this uses the juju4.lxd Ansible role to install LXD on a Ubuntu or RedHat-based Linux system. Other distributions might work, but are not tested. Also, note the --user markus command-line argument. This lets you specify one or several users who'll have permissions to use the LXD service on this host. After running this, you'll have to either log-out of your current session and log-in again, or execute newgrp lxd to take advantage of this permission.

It is possible to adjust some parameters related to your LXD setup with this frecklet (like for example network addresses or storage settings), but this is out of scope in this context.

Describing the LXD-image contents

We'll be writing a relatively simple frecklet, not different at all to anything we'd write for a different target type (e.g. a remote server). For more details on how to all this works, and how you can write your own, please check the freckles getting started guide and the other frecklet-related documentation.

For our example, lets install the Nginx webserver and configure it to host a single, static html site (which we also create dynamically).

As this is a relatively 'normal' thing to do, there are already pre-made frecklets that can help us implement the required steps, which are:

• install the Nginx packages and enable/start the service (nginx-service)
• create a configuration for our vhost (nginx-vhost-from-folder)
• create our html file (file-with-content)

Here's how that looks as a frecklet:

- nginx-service
- nginx-vhost-from-folder:
    document_root: /var/www/html
    default_server: true
    use_https: false
- file-with-content:
    owner: www-data
    path: /var/www/html/index.html
    content: |
      <h1><i>freckles</i> says "Hello World", from {{:: from ::}}!</h1>

Note the {{:: from ::}} template string in our frecklet code: this lets us customize the generated html for every LXD image (if we chose to do so).

To test, lets launch an LXC ocntainer and provision it with our new frecklet. We could do this with a simple lxc launch ... command, but while we're at it let's use freckles for this too. In an interactive session that doesn't make much sense, since it's also just one command, and comes with a bit of overhead. But it's a good opportunity to show how one would do that if launching an LXD container was part of a provisioning pipeline. So, here goes:

$ frecklecute lxd-container-running --image-name 'debian/10' \
     --image-server https://images.linuxcontainers.org \
     --name 'freckles-test' \
     --register-addresses freckles_container_ip

╭╼ starting run
│  ├╼ running frecklet: lxd-container-running (on: localhost)
│  │  ├╼ starting Ansible run
│  │  │  ├╼ create/launch container 'freckles-test'
│  │  │  │  ╰╼ ok
│  │  │  ╰╼ ok
│  │  ╰╼ ok
│  ╰╼ ok
╰╼ ok

Result:

    freckles_container_ip:
      eth0:
      - 10.10.10.42

In this example, we provided the optional --register-addresses command-line argument. That prompts freckles to register the (network) details of our newly created container into a variable, which makes it easier for us to connect to it later. Alternatively, you can get the ip address with a simple lxc list.

Now that our container is running, we can provision it using our frecklet from above. We'll save the content string as example-lxd.frecklet, then issue:

$ frecklecute --target lxd::freckles-test example-lxd.frecklet --from 'a test run'

The crucial thing is the --target lxd::<container_name> part. It tells freckles to not connect via ssh, but directly via 'lxd'.

This will take a minute or two, as there is some bootstrapping to be done in addition to installing Nginx. Once the process is finished, you should be able to point your browser to the IP address displayed in the run before this, and see the message "freckles says "Hello World", from a test run!"

Sweet, now we have everything we need to make that into an LXD container image...

Building an LXD image

While it's possible to create an LXD image from scratch using freckles, for example by capturing the steps from this tutorial into a frecklet, we won't do that here. We are going to be cheating a bit, and use Packer instead, since that offers the same functionality (and more), with less hassle.

As we already have our example-lxd.frecklet file, all we need to do is provide it's full path in the following command:

$ frecklecute container-image-from-frecklet-file \
     --install-packer \
     --image-type lxd \
     --source-image images:debian/10 \
     --image-name freckles-image \
     --frecklet-path $(pwd)/example-lxd.frecklet \
     --frecklet-vars '{"from": "my first image"}'

Let's look at that command in detail:

• container-image-from-frecklet-file: the name of the frecklet we want to execute (we could have also used the wrapper frecklet lxd-image-from-frecklet-file
• --install-packer: we make sure that the Packer executable is available on our system (using the packer-installed frecklet under the hood)
• --image-type lxd: we want an LXD image (we could have also said: docker, but since there is some systemd/init stuff in our frecklet, indirectly, that won't work in this case)
• --source-image images:debian/10: the source image to use (notice the 'images:'-prefix, this is needed in this case to use the correct LXD image index
• --image-name: the name of our new image
• --frecklet-path $(pwd)/example-lxd.frecklet (absolute!) path to the frecklet that contains our provisioning instructions
• --frecklet-vars '{"..."}': additional variables (as a JASON string) which are needed by our provisioning frecklet. The alternative to using this is hard-coding everything in the frecklet file

So, once this command finished, you should be able to see the new image via:

$ lxc image list
+----------------+--------------+--------+-----------------------------------------------+--------+-----------+-------------------------------+
|     ALIAS      | FINGERPRINT  | PUBLIC |                  DESCRIPTION                  |  ARCH  |   SIZE    |          UPLOAD DATE          |
+----------------+--------------+--------+-----------------------------------------------+--------+-----------+-------------------------------+
| freckles-image | f9600c979eba | no     | built with freckles.                          | x86_64 | 194.72MB  | Aug 10, 2019 at 10:37am (UTC) |
...
...

And we can use it like so:

$ frecklecute lxd-container-running --image-name 'freckles-image' \
      --name 'freckles-example' --register-addresses freckles_container_ip

╭╼ starting run
│  ├╼ running frecklet: lxd-container-running (on: localhost)
│  │  ├╼ starting Ansible run
│  │  │  ├╼ create/launch container 'freckles-example'
│  │  │  │  ╰╼ ok
│  │  │  ╰╼ ok
│  │  ╰╼ ok
│  ╰╼ ok
╰╼ ok

Result:

    freckles_container_ip:
      eth0:
      - 10.10.10.100

Now point your browser to the container ip (10.10.10.100 in my case), and you should see a message like: 'freckles says "Hello World", from my first image!'.

Re-using our frecklet

The main reason I wrote freckles was to have a generic, flexible and re-usable way to describe desired states of computational environments. We already used our frecklet to provision a running (empty) LXD container, and to create an LXD container image from it. If our frecklet didn't contain init-system-specific instructions (setting up and restarting a systemd service unit), we could also use it to build a Docker image. This won't work here, but lets use it instead to setup a remote server (like an EC2 instance, or a Digital Ocean droplet):

Requirements:

• (empty) Ubuntu/Debian remote machine with public IP
• root or sudo access to that machine

We don't need to make any changes to our frecklet, all we need to do is change the --target in our command:

$ frecklecute -t root@159.69.201.220 example-lxd.frecklet --from "a remote machine"

╭╼ starting run
│  ├╼ running frecklet: /home/markus/projects/frkl-dev/frkl.io/example-lxd.frecklet (on: 159.69.201.220)
│  │  ├╼ starting Ansible run
│  │  │  ├╼ updating apt cache
│  │  │  │  ╰╼ ok
│  │  │  ├╼ ensure rsync, ca-certificates and unzip packages are installed
│  │  │  │  ╰╼ ok
│  │  │  ├╼ creating freckles share folder
│  │  │  │  ╰╼ ok
│  │  │  ├╼ creating box basics marker file
│  │  │  │  ╰╼ ok
│  │  │  ├╼ recording python interpreter metadata
│  │  │  │  ╰╼ ok
│  │  │  ├╼ recording box metadata for later runs
│  │  │  │  ╰╼ ok
│  │  │  ├╼ Ensure nginx is installed.
│  │  │  │  ╰╼ ok
│  │  │  ├╼ Copy nginx configuration in place.
│  │  │  ├╼ reload nginx
│  │  │  │  ╰╼ ok
│  │  │  ├╼ write content to file: /etc/nginx/sites-enabled/localhost.http.conf
│  │  │  │  ╰╼ ok
│  │  │  ├╼ ensure user 'www-data' exists
│  │  │  │  ╰╼ ok
│  │  │  ├╼ write content to file: /var/www/html/index.html
│  │  │  │  ╰╼ ok
│  │  │  ├╼ geerlingguy.nginx : reload nginx
│  │  │  │  ╰╼ ok
│  │  │  ╰╼ ok
│  │  ╰╼ ok
│  ╰╼ ok
╰╼ ok

$ curl http://159.69.201.220
<h1><i>freckles</i> says "Hello World", from a remote machine!</h1>%