Docker Compose Alternatives for Kubernetes: Skaffold

By Rich Burroughs

If you're developing apps that run in Kubernetes, running them locally with Docker Compose may seem like a simple solution. But it can cause problems, as your local environment will be very different from how your apps run production. Developing against a local Kubernetes cluster (like Docker Desktop or Minikube) can help address this issue, but it can be slow to build and deploy containers to Kubernetes every time you make a change in your code.

Luckily there's a class of tools that help make these local workflows easier. In this series, we'll be looking at some of those tools and their features. Our previous post in this series was about Tilt. In this post, we'll be looking at Skaffold, which Google created. Another popular tool for development workflows is called DevSpace. I work at Loft Labs, the company that created DevSpace, and I have a big interest in developer productivity. In another one of my articles, you can learn more about DevSpace as another Docker Compose alternative for Kubernetes. But in this post, we are going to take a look at what Skaffold has to offer.

Let's take a look at Skaffold, starting with the installation.

Installation

I use an M1 Mac, and the Skaffold installation instructions for macOS consist of a curl command piped to bash. This is what I ran for my install:

curl -Lo skaffold https://storage.googleapis.com/skaffold/releases/latest/skaffold-darwin-arm64 && \ sudo install skaffold /usr/local/bin/

Skaffold is also available in Homebrew for people using Macs. Additionally, there are instructions for installing on Linux, Windows, with "docker run" and Google Cloud's gcloud command.

Quickstart

The Skaffold Quickstart requires having Skaffold, Minikube and kubectl installed. There's also a link to run a Google Cloud Shell with all of those apps preinstalled. That seems super handy, but I wonder whether some users who are using other cloud providers might not feel left out when there's a link to a Google Cloud Platform (GCP) option but there's not one for their cloud. I can understand Google wanting to enable GCP users, but this is an open source project and these lines can get a little blurry.

The quickstart uses a simple Go app that prints "Hello World" to the terminal every second, which is a very clear way to illustrate what's happening. To get started in the development mode, you type:

skaffold dev

That builds the app and runs it. When you make changes in the main.go file and save them, you'll see the app rebuild and your new code will run.

You can also build and deploy the app just once with this command:

skaffold run

The skaffold.yaml File

The apps in this space seem to all have a central configuration file that defines the workflow. In the cast of Tilt, it's a file written in Starlark. With Skaffold, it's a YAML file. Here's the example from the quickstart:

apiVersion: skaffold/v2beta17
kind: Config
build:
 artifacts:
 - image: skaffold-example
deploy:
 kubectl:
   manifests:
     - k8s-*

The Kubernetes manifest it's referring to in the deploy section looks like this:

apiVersion: v1
kind: Pod
metadata:
  name: getting-started
spec:
  containers:
  - name: getting-started
    image: skaffold-example

Skaffold will rebuild the getting-started container anytime changes are saved to the main.go file. It builds the container and then deploys it using kubectl. Skaffold also supports deploying with Helm or kustomize.

Skaffold.yaml files can be much more complex than the one in this example. There's a reference doc here.

File Sync

Rebuilding and deploying containers on every saved change works fine with a small Go project, but it can be slower and more complex with more significant projects. One of the features that the tools in this space all share is what's referred to as hot reloading or live reloading - the ability to update changes files in the running container instead of building and deploying it every time something changes. With Skaffold, this is done using a feature that's in beta called File Sync.

File Sync is configured in skaffold.yaml and can run in one of three modes: Manual, where you specify the files that will be updated on sync; Inferred, where Skaffold uses the Dockerfile to determine what to sync for Docker images; and Auto mode, where Skaffold looks for known file types to sync.

Both the manual and inferred modes require a bit more configuration in the skaffold.yml. Auto mode only seems to be available if you're using jib (for Spring Boot projects) or Cloud Native Buildpacks.

I found Skaffold's File Sync configuration a bit harder to get a handle on than Tilt's approach. But once you do it for a project, you won't likely need to change it often.

Skaffold's Pipeline

Here's a look at the Skaffold pipeline from the docs:

There are instructions for building with several methods (Docker, Jib, Bazel, etc.) and deploying with kubectl, kustomize, or Helm.

Skaffold seems to integrate with several popular tools, but I didn't see an equivalent to Tilt's extensions. Being able to leverage people's work in the community for these kinds of integrations is a big plus.

I appreciated that there was documentation focused on the testing stage, using Container Structure Tests or custom tests that you write. There's also a section in the docs explaining debugging with the "skaffold debug" command.

Conclusion

Skaffold is much more of a Command Line Interface (CLI) oriented tool than Tilt, which focuses more on its web UI. It's simply a matter of preference for you and your team. I have a lot of CLI experience and tend to lean that way, but both approaches have advantages and disadvantages.

Skaffold seems to solve many problems that people have when they develop against a Kubernetes cluster. It would be an excellent choice for many teams, especially people using other Google tools like Google Cloud Platform and kustomize.

If you are currently using Docker Compose and your apps run in Kubernetes clusters, you should look into the tools in this space and see which one works best for your team.