Building multi arch Docker images without using qemu

Lately I've been doing lots of Rust with MRML, Catapulte and Jolimail, and one of the pain points of Rust is that the compiling process is pretty slow. When, on top of that, you want to build it for amd64, i386, arm64 and arm32v7 because your application is open source and supposed to work everywhere, it becomes a bit tricky to make it build.

Candidly, I initially tried to build it using docker and Qemu for multiple architectures but I finished with some weird errors.

After a bit of chatting with my previous colleagues from Docker, I was encouraged to try to cross compile. Cross compiling is really great when you don't have a lib dependency on the system. But Catapulte and Jolimail are based on actix which depends on OpenSSL, which is supposed to be compiled for the architecture. In short, to cross compile your rust code you first have to cross compile OpenSSL. If you have other libraries, well, cross compile them.

So I did it for Jolimail, it cross compiles, but deep down, I don't like this approach for several reason. First, I've to handle the build of OpenSSL. If there is a new version, I've to update everything and not only rebuild my image. Then, it adds an unwanted complexity to my Dockerfile. So I decided to find another way to do this.

After a bit of digging and a good hint from a nice user on the docker slack community (Billy), I finally did my first use of docker buildx create --append. If you're not familiar with docker buildx and multi arch images, I encourage you to take a look at the article I wrote, back when I was at Docker.

When you are on an amd64 machine and you want to make a multi arch image, the initial buildx solution is to use Qemu to emulate the target environment. But you can also, when you create a buildx context, specify a context target for a specific platform.

Like lots of my peers, I bought every version of the raspberry pi, I even have several of each version, but I don't really have time to properly use them. So I started one (3b+ I guess) and installed HypriotOS on it (take a look, install it, test it, it's great). I then created a context to be able to build on the raspberry pi from my local machine.

docker context create rpi --docker "host=ssh://pirate@black-pearl"

When this is done, I am able to target the raspberry pi by doing DOCKER_CONTEXT=rpi docker pull jdrouet/kaamelott-quote.

Then the trick is to tell buildx that, when it needs to build the image for arm32v7, it should use the context rpi.
First, I use docker buildx ls to get the name of the current builder. Then, I append the context to this builder with the following command.

docker buildx create --append --name you-builder-name --platform linux/arm/v7 rpi

And that's it! I can now build your multi arch image.

docker buildx build --tag jdrouet/catapulte:some-tag --platform linux/amd64,linux/i386,linux/arm/v7 .