Build docker images inside of Kubernetes

There are many reasons why we might want to build docker images inside of a Kubernetes cluster. Reasons are out of scope of this post but one reason might be to integrate them in you CI pipelines.

Build and push docker images inside of k8s as you please, on the other hand running containers can be very problematic. It is problematic because k8s will not know about these containers and will not be able to manage them. This just defeats the purpose of using k8s in the first place.

That said if you really need to run unmanaged containers on your own, consider creating a separate node pool just for that purpose.

Let us talk requirements, our requirements are basic:

• We want to be able to build docker images in a specific pod.
• Push images to a local registry or docker hub.
• Caching would be nice-to-have, otherwise each are build would take a long time.
• Finally, preferably we will use a daemonless docker build tool instead of installing docker inside a container and mount /var/run/docker.sock

A google search for "Daemonless docker builds" will return a few results like:

• buildah
  Buildah's commands replicate all of the commands that are found in a Dockerfile. This allows building images with and without Dockerfiles while not requiring any root privileges. Buildah follows a simple fork-exec model and does not run as a daemon but it is based on a comprehensive API in golang, which can be vendored into other tools. *
  
  Here is a list of the commands relevant to this post:
  

  # Build an image using instructions from Dockerfiles.
  buildah bud  
  
  # Push an image from local storage to elsewhere.
  # `--tls-verify=false` is used to allow using insecure registries if required.
  buildah push  
  
  # List all images built by buildah.
  buildah images  
  

  More examples are available here

• img: https://github.com/genuinetools/img

• Kaniko: https://github.com/GoogleContainerTools/kaniko

• dind (Docker in Docker): https://github.com/jpetazzo/dind

• buildkit: https://github.com/moby/buildkit/tree/master/examples/kubernetes

• Sysbox: https://github.com/nestybox/sysbox-external

• Knative build (deprecated): https://starkandwayne.com/blog/build-docker-images-inside-kubernetes-with-knative-build/

After all the research, I landed on using buildah with an insecure registry. Here are a few commands I used for testing

# debug building a dockerfile
buildah --debug bud -f Dockerfile -t mostafagazar/test .

# Without build args
buildah bud -f Dockerfile -t mlstudio-registry.default.svc.cluster.local:5000/test:v0 .
buildah push --tls-verify=false mlstudio-registry.default.svc.cluster.local:5000/test:v0

# With build args
buildah bud --build-arg model_name=name -f Dockerfile -t mlstudio-registry.default.svc.cluster.local:5000/test2:v0 .
buildah push --tls-verify=false mlstudio-registry.default.svc.cluster.local:5000/test2:v0

Want to get early access to ML Studio and product updates? Subscribe here or follow me on twitter