Docker is really a great tool and provides a convenient way to package and deploy applications.
Until my recent experience, I believed that Docker images were portable and could run on any machine regardless of CPU architecture. Basically, one of my colleagues built an image on his Mac and shipped it to me, and I was attempting to run it on Ubuntu then I realised I was mistaken.
The problem was that Mac's use ARM processors, whereas mine was AMD processor and whenever I attempted to run the container, it displayed a warning and the container was abruptly terminated.
WARNING: The requested image's platform (linux/arm64) does not match the detected host platform (linux/amd64) and no specific platform was requested
Later, after doing some research, I discovered something called docker buildx with --platform flag, which we can use while building an image.
Docker Buildx
Docker Buildx is a command-line tool that provides advanced capabilities for building and managing multi-platform Docker images. Basically, it is an extension to the regular docker build command.
Using docker buildx, we can create docker images that are compatible with multiple CPU architectures, such as AMD64, ARM64, etc. Honestly, after encountering the above issue, I think it is pretty useful when there is a requirement for deploying or running a container on different architectures.
A quick rundown of how it can be utilised
- Builder instance: For using
docker buildx, there should be a builder instance up and running and for creating a builder instance firedocker buildx create --use.
$ docker buildx create --use --name buildx_instance
buildx_instance
Once the builder instance is created and active, it can be used using
docker buildx buildcommand for building images.As well the status of the instance can be checked using
docker buildx lsordocker pscommand.
$ docker buildx ls
NAME/NODE DRIVER/ENDPOINT STATUS BUILDKIT PLATFORMS
buildx_instance * docker-container
buildx_instance0 unix:///var/run/docker.sock running v0.11.6 linux/amd64, linux/amd64/v2, linux/amd64/v3, linux/386
default docker
default default running 20.10.22 linux/amd64, linux/386
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
e023022a7b0e moby/buildkit:buildx-stable-1 "buildkitd" 6 minutes ago Up 6 minutes buildx_buildkit_buildx_instance0
-
docker buildx buildcommand can be used for building images, the intresting option while building the image is--platformflag. In this command we are creating a image which is compatible with ARM as well as AMD processors.
$ docker buildx build --platform linux/amd64,linux/arm64,linux/arm/v7 -t sunnybhambhani/multi_test:v2 .
- Note: The images won't be visible under
docker imagesand these images will be in buildx cache only. Further--pushflag can be used to directly push the images to dockerhub.
$ docker buildx build --platform linux/amd64,linux/arm64,linux/arm/v7 -t sunnybhambhani/multi_test:v2 --push .
- Here is how it will be visible on dockerhub:
Additionally, docker buildx imagetools inspect can be used for inspecting docker images.
$ docker buildx imagetools inspect sunnybhambhani/multi_test:v2
Name: docker.io/sunnybhambhani/multi_test:v2
MediaType: application/vnd.docker.distribution.manifest.list.v2+json
Digest: sha256:af1c773ab325739067687e2df19137de675b6c7b4d618e6fac0197b55d8136bc
Manifests:
Name: docker.io/sunnybhambhani/multi_test:v2@sha256:1b270f007c1645e69e43595f0b5cd7fd06951c2974901943bcc503647b4fb35f
MediaType: application/vnd.docker.distribution.manifest.v2+json
Platform: linux/amd64
Name: docker.io/sunnybhambhani/multi_test:v2@sha256:4520b7bb8af2c868205524856667aa88ce180d260330aef3a325f69a3f336acb
MediaType: application/vnd.docker.distribution.manifest.v2+json
Platform: linux/arm64
Name: docker.io/sunnybhambhani/multi_test:v2@sha256:1f5d06b9181d8e4b5b552f2c5f5f0978d9409aef5233059c3136b469871c5b7f
MediaType: application/vnd.docker.distribution.manifest.v2+json
Platform: linux/arm/v7
Other important docker buildx commands:
docker buildx du: Check the disk usage
docker buildx imagetools inspect: To inspect the created images
docker buildx inspect: To inspect current builder instance
docker buildx ls: To list builder instances
docker buildx prune: To remove build cache
docker buildx rm: To remove a builder instance
docker buildx stop: To stop builder instance
References and more details around this topic can be found on below URLs:
- https://docs.docker.com/build/building/multi-platform/
- https://github.com/docker/buildx
docker buildx --help
Feel free to add your thoughts and correct me if I am wrong or have missed something, Happy learning :)
Top comments (5)
This is really useful to know, I didn't ecounter the experience yet but I will keep it in mind
Readers may want to check Using Docker manifest to create multi-arch images for more multi-arch docker info.
Unfortunatelly on Apple Silicon Macs those linux/amd64 images will run only in 32 bit CPU op-mode at the time of writing.
Especially if you try running some libs with native extension (as its often in python) those libs wont work.
so in case you came here as me, to find a way to test code locally under linux/amd64 emulation on a mac , this method wont work for most of things at the time of writing
I think the "FROM" directive in Dockerfile chooses the right CPU architecture while building. If we are to transfer that image to a different CPU type/architecture PC, and create container, it will fail since the container now has to run as a process in the host OS, which is incompatible. This is similar to running a binary file on an incompatible CPU architecture, such as ARM-compiled binary on Intel PC.
Like we recompile the binary again to match the target CPU architecture, we'll have to "build" the docker image again. The "buildx" command here is equivalent to cross-compilation, and I'm afraid may increase image size considerably if we're including more than one architecture in the command.
When we say Docker is portable, it's within the context of what's inside the container - base os, application runtime and application. The docker image itself is OS-specific and require kernel facilities such as cgroups and namespaces, in order to instantiate container (process).
All these constraints when we think about it, it's because "docker image is immutable" and can't have runtime hooks changing its behaviour.