Security is arguably one of the largest issues in Kubernetes environments. Other than the fact that almost all engineers are still trying to figure out Kubernetes, once it’s figured out, the next concern is security.
Red Hat’s most recent report states that 70% of security issues are Kubernetes misconfigurations. Typically, misconfigurations are due to having poor policies or no policies in place.
That’s where a platform like Kyverno comes into play.
In this blog post, you’re going to learn about Kyverno, why it’s important, how it’s compared to Open Policy Agent (OPA), and how you can get started with it in production today.
When you deploy anything to Kubernetes, whether it’s a Pod, ConfigMap, or an entire containerized microservices, you’ll typically want some type of rules around them. For example, one of the best practices that every engineer should implement is to never use the
latest tag of a container image inside of a Kubernetes Manifest in production. Why? Because the
latest tag is typically a dev build that’s still being worked out. Instead, engineers should use a specific version of the containerized app.
Here’s the thing - how do you block engineers from using the
That’s what a policy manager like Kyverno does.
Kyverno manages policies, whether they’re for security or general best practices, in your Kubernetes environment. You can set up policies for something like the
latest tag example above, or something more security-centric like verifying container images for a software supply chain security. Kyverno works outside of standard Kubernetes Manifests as well. You can use overlays like Kustomize or Helm and implement Kyverno policies.
Open Policy Agent (OPA) is gaining increased popularity in the policy management community, primarily because it’s vendor agnostic. Meaning, you can create policies for any environment, not just Kubernetes.
With Kyverno, it’s only for Kubernetes.
If you need policy management outside of Kubernetes, Kyverno won’t work and you may be better off going the OPA route.
What should be said for Kyverno, which is a major upside over OPA, is that Kyverno policies are created via Kubernetes Manifests. OPA policies are created with Rego, a configuration language that sort of looks like Go (golang). With that being said, if you don’t already know Rego, implementing OPA may be a burden. Where-as with Kyverno, you’re already used to Kubernetes Manifests, so it’s less of a lift.
A policy manager like Kyverno or OPA are Kubernetes Controllers that run inside of a Kubernetes cluster. Because of the underlying features of a Kubernetes Controller, like managing the state of deployed resources, it makes sense for a policy manager to be a Controller. The whole idea is to manage policies, and you can’t manage policies without proper state in place.
With policy managers, it’s all about webooks. There are policies in place inside of the Kubernetes cluster and whenever a request is made, like creating a Kubernetes Deployment resource, the webhook is received and makes a callback to the Kubernetes API server. It then checks to confirm that all of the policies in place are passed and if they are, the Kubernetes resource will be created. If they aren’t, the resource will be blocked.
When you deploy Kyverno in a Kubernetes cluster, a few Kubernetes resources are created:
- A Namespace where Kyverno will live
- The Kyverno CRD’s
- The Kyverno container image
- The Kyverno services
- A service account with cluster roles and rolebindings. Please note that the service account does contain
You can find the entire Kubernetes configuration for more detail here: https://raw.githubusercontent.com/kyverno/kyverno/main/config/install.yaml
Now that you know what the Kubernetes configuration consists of, let’s look at how to install and configure Kyverno.
First, run the Kubernetes Manifest installation.
kubectl create -f https://raw.githubusercontent.com/kyverno/kyverno/main/config/install.yaml
If you’d prefer to use a Helm Chart, use the code below.
helm repo add kyverno https://kyverno.github.io/kyverno/ helm repo update helm install kyverno kyverno/kyverno -n kyverno --create-namespace
For production purposes and ultimately what you want your environment to look like, take a look at the documentation found here.
After installing Kyverno, you should see the following output on your terminal.
Run the following command to see all of the resources in the Kyverno namespace.
kubectl get all -n kyverno
Now that Kyverno is installed, let’s take a look at policy configurations.
Luckily, Kyverno already has a ton of policies that are available. As mentioned in one of the previous sections, they’re all written in YAML as a Kubernetes Manifest. You can see a ton of the policies here.
For the purposes of this configuration, let’s use the policy called Disallow Latest Tag, which blocks a Kubernetes resource from being deployed with the
latest container image.
Open up a new file and call it
disallow.yaml, and paste in the following policy.
apiVersion: kyverno.io/v1 kind: ClusterPolicy metadata: name: disallow-latest-tag annotations: policies.kyverno.io/title: Disallow Latest Tag policies.kyverno.io/category: Best Practices policies.kyverno.io/severity: medium policies.kyverno.io/subject: Pod policies.kyverno.io/description: >- The ':latest' tag is mutable and can lead to unexpected errors if the image changes. A best practice is to use an immutable tag that maps to a specific version of an application Pod. This policy validates that the image specifies a tag and that it is not called `latest`. spec: validationFailureAction: audit background: true rules: - name: validate-image-tag match: resources: kinds: - Pod - Deployment validate: message: "Using a mutable image tag e.g. 'latest' is not allowed." pattern: spec: containers: - image: "!*:latest"
If you looked at the policy via the link above for Disallow Latest Tag, you’ll notice that the
kinds used to be
Pod. The code was changed to
Pod instead of just
Pod, which is why what’s on the Kyverno website looks different in the Kubernetes Manifest above.
Apply the Manifest to your Kubernetes cluster.
kubectl create -f Kyverno/disallow.yaml clusterpolicy.kyverno.io/disallow-latest-tag created
Next, create a new YAML file called
nginx.yaml and paste in the following configuration.
apiVersion: apps/v1 kind: Deployment metadata: name: nginx-deployment spec: selector: matchLabels: app: nginxdeployment replicas: 2 template: metadata: labels: app: nginxdeployment spec: containers: - name: nginxdeployment image: nginx:latest ports: - containerPort: 80
Notice how it’s using the
latest tag of the Nginx container image.
nginx.yaml Manifest and you should see an output similar to the below.
kubectl create -f nginx.yaml Error from server: error when creating "nginx.yaml": admission webhook "validate.kyverno.svc-fail" denied the request: policy Deployment/default/nginx-deployment for resource violation: require-labels: autogen-check-for-labels: 'validation error: label ''app.kubernetes.io/name'' is required. rule autogen-check-for-labels failed at path /spec/template/metadata/labels/app.kubernetes.io/name/'
Congrats! You have officially set up Kyverno policies for Kubernetes.