When you’re trying to figure out a folder hierarchy for your apps or how you’re going to pass in variables, parameters, and secrets, it can get tough. Especially if you have multiple applications and environments that you need to account for and manage throughout lifecycles.
With Kustomize, things get a little bit easier to manage.
In this blog post, you’ll learn about what Kustomize is, why you’d want to use it, and how to get started with it today.
Every application that you have will have to run in multiple environments. Typically, you’ll see something like Development, Staging (or UAT), and Production. However, there can be plenty of other environments, like QA. If you have a multiple Kubernetes Manifests, that means you’ll have to create each Manifest for each environment. For example, If you have a Deployment.yaml and a Service.yaml Kubernetes Manifest, that means you’ll have to create them for each environment as things like volume mounts, replicas, names, etc. will be different. This is painful and a hassle because:
- Managing multiple manifests is a pain
- You’ll have to comb through multiple files for each change, like a container image version change
- A ton of tech debt can be accrued
with Kustomize, it’s a better different.
Kustomize allows you to have one file that contains values for every environment, and instead of having to have multiple Kubernetes Manifests, you simply point to the same Kubernetes Manifest for every environment. The only thing that changes are the values in each
There are a ton of different use-cases for Kustomize, but you may be wondering to yourself why you’d use it over, say, an environment variables (
.env) file or something along those lines.
The first thing is having environment files and managing them for each environment as a separate entity can cause confusion for engineers down the line and ultimately result in tech debt. Kustomize is a Custom Resource that was created to work with the Kubernetes API, so the syntax for creating the files is the same as any other manifest. The CRD is
The next big thing is because Kustomize is part of the Kubernetes API, it works with
kubectl, which means you don’t need to have another tool in your toolbelt. You can keep the configurations close to the Kubernetes API instead of looking at outside resources.
The third is when thinking about multiple environments, like Development, Staging, and Production, it’s great to have some segregation. Having the ability to split up environments for each application not only saves you time later when figuring out what goes where in your config, but it also allows you to easily use tools/platforms like Helm Charts for multi-environment configurations and packaging up Kubernetes manifests. When thinking about Helm Charts, you won’t need a config for each environment. Instead, you’d have one set of Helm Charts that’s reading from Kustomize configurations.
Now that you know about Kustomize and why you’d want to use it, let’s take a look at how to use it from a hands-on perspective.
First, let’s create two Kubernetes Manifests going off of the same file structure in the above screenshot from the What Is Kustomize section.
apiVersion: apps/v1 kind: Deployment metadata: name: nginx-deployment spec: selector: matchLabels: app: nginxdeployment replicas: 2 template: metadata: labels: app: nginxdeployment spec: containers: - name: nginxdeployment image: nginx:latest ports: - containerPort: 80
apiVersion: v1 kind: Service metadata: name: nginxservice spec: selector: app: nginxdeployment ports: - protocol: TCP port: 80 type: LoadBalancer
Next, in the
base directory, create a file called
kustomization.yaml and add in the following:
apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization resources: - deployment.yaml - service.yaml
resources list tells Kustomize what the base configuration is.
Next, create a new directory structure outside of the
base directory that goes
overlays -->dev. It should look like the screenshot below.
Create a new file called
kustomization.yaml (this is different from the
kustomization.yaml file in the
base directory) and add in the following Manifest:
apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization resources: - ../../base/ replicas: - name: nginx-deployment count: 1
What the above Manifest is saying is:
- The base
service.yamlis in the
basedirectory. Look there for the Kubernetes Manifests.
- For the
devenvironment, ensure that only 1 replica is deployed.
Now that you have the proper directory structure and Manifest configurations, you can run the Kustomize setup.
To run a Kustomize setup, do it in the same way that you’d deploy a Kubernetes Manifest except:
- Instead of using
-f, you would use
- Specify the directory that the
kustomization.yamlfile exists in instead of the actual file itself.
For example, the below command specifies the
kustomization.yaml Manifest is in the current directory.
kubectl apply -k .
If you have the same folder structure that you saw in the What Is Kustomize section, you’d want to
cd into the
dev directory and run
kubectl apply -k .
Running the Kustomize configuration from the
dev directory, because you’re specifying that 1 replicas should exist, you’ll see five Pods running like in the screenshot below.
Although Kustomize definitely makes for far more file management, if you’re already using
.env files or another form of variable/parameter files, Kustomize will be a breath of fresh air. You can set a
kustomize.yaml for each environment, put in the parameters/variables/config data that you want for each environment, and get a deployment going for any stage in your development cycle.