Using Kyverno Policies for Kubernetes Governance

Kyverno is a great tool that can be installed into any Kubernetes cluster, allowing cluster administrators to enforce policies for resources in the cluster, and even modify resources before they are applied. Kyverno can be used to ensure deployments are secure, ensure deployments meet certain organizational criteria (e.g. define a cost center label), or even ensure all deployments mount a common volume.

Kyverno works by deploying a pod and services into your existing cluster. It creates multiple Admission Webhooks in the cluster. These webhooks are responsible for handling API requests coming in to Kubernetes and either validating something from the request (Validating Admission Webhook, or modifying the request before it is applied (Mutating Admission Webhook).

In the diagram below, you can see where in the process of an API call to Kubernetes, each of the Mutating Admission and Validating Admission webhooks will run.

All of the Kubernetes Manifests and Kyverno Policies are available in this GitHub repo.

Deploying Kyverno

Getting started with Kyverno is pretty simple. While there are a lot of knobs that can be turned to configure Kyverno, the initial default goes a long way. I deployed this into my DigitalOcean Kubernetes cluster.

To install, I simply ran:

$ helm repo add kyverno https://kyverno.github.io/kyverno/
$ helm repo update
$ helm install kyverno kyverno/kyverno --namespace kyverno --create-namespace

I now have a single Kyverno pod running in the cluster:

$ kubectl get pods -n kyverno
NAME                       READY   STATUS    RESTARTS   AGE
kyverno-6d94754db4-tdl9s   1/1     Running   0          5s

Policies

Anatomy of a basic policy

Policies can be written with many different options in Kyverno. The most basic policies can be written to check values of specific field(s) within an API request to Kubernetes, and make a decision whether the request should be allowed or not.

Require specific pod labels

This example requires specific labels to be set on a pod prior to creating those resources. One of the great benefits Kyverno provides is that you can specify Pod as the resource kind, but it will also check against the policy whenever creating resources which will end up creating pods (Deployment, StatefulSet, etc.)

The below example policy requires that all pods have the labels.acmecorp.com/costCenter and labels.acmecorp.com/department labels. They can be set to any value.

apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: require-business-labels
  annotations:
    policies.kyverno.io/title: Require Business Labels
    policies.kyverno.io/category: Best Practices
    policies.kyverno.io/severity: medium
    policies.kyverno.io/subject: Pod, Label
    policies.kyverno.io/description: >-
      Define required labels used by our internal business processes to understand which applications
      are running in each cluster, and used to handle chargeback activities for resources consumed
      by this specific application.
spec:
  validationFailureAction: enforce
  background: false
  rules:
  - name: check-for-business-labels
    match:
      resources:
        kinds:
        - Pod
    validate:
      message: "The labels `labels.acmecorp.com/costCenter` and `labels.acmecorp.com/department` are required."
      pattern:
        metadata:
          labels:
            labels.acmecorp.com/costCenter: "?*"
            labels.acmecorp.com/department: "?*"

Example Policy Tests:

• Example Deployment which will be denied under this policy.
• Example Deployment which will be allowed under this policy.

Demo showing policy in cluster:

Require the use of a specific container registry

This policy checks for 2 important items: 1. images specified in a pod definition must be from a specific container registry (in this case, from the DigitalOcean registry); and 2. images cannot have the latest tag.

apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: restrict-image-registries
  annotations:
    policies.kyverno.io/title: Restrict Image Registries and Latest
    policies.kyverno.io/category: Best Practices
    policies.kyverno.io/severity: medium
    policies.kyverno.io/subject: Pod
    policies.kyverno.io/description: >-
      Requires all images for pods be sourced from the Digital Ocean Container Registry. Any other
      image sources are denied.
spec:
  validationFailureAction: enforce
  background: false
  rules:
  - name: validate-registries
    match:
      resources:
        kinds:
        - Pod
    validate:
      message: "Unknown image registry."
      pattern:
        spec:
          containers:
          - image: "registry.digitalocean.com/*"
  - name: validate-image-tag
    match:
      resources:
        kinds:
        - Pod
    validate:
      message: "Must not use the tag `latest` on any images."
      pattern:
        spec:
          containers:
          - image: "!*:latest"

Example Policy Tests:

• Example Deployment which will be denied under this policy.
• Example Deployment which will be allowed under this policy.

Demo showing policy in cluster:

Require a runAsUser be specified

This policy requires that every container specified within a pod has a runAsUser defined and that the value is greater than 0, meaning the container cannot run as root. In reality, there are additional items you will probably want to check in a policy like this, but this provides a good place to get started.

apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: require-pod-runasuser
  annotations:
    policies.kyverno.io/title: Require the RunAsUser to be Specified
    policies.kyverno.io/category: Best Practices
    policies.kyverno.io/severity: medium
    policies.kyverno.io/subject: Pod
    policies.kyverno.io/description: >-
      Requires Pods to specify as runAsUser value within their containers which are not root.
spec:
  validationFailureAction: enforce
  background: false
  rules:
  - name: check-userid
    match:
      resources:
        kinds:
        - Pod
    validate:
      message:  >-
        The field spec.containers.*.securityContext.runAsUser must specified and greater than zero.
      pattern:
        spec:
          containers:
          - securityContext:
              runAsUser: ">0"

Example Policy Tests:

• Example Deployment which will be denied under this policy.
• Example Deployment which will be allowed under this policy.

Demo showing policy in cluster:

Conclusion

Overall, this is meant to be a couple basic examples of using policies with Kyverno in Kubernetes. You can certainly get much more complex, or implement a policy which mutates a resource, to keep resources in your cluster compliant with your rules.

If you end up trying Kyverno and find out you need something more complex, take a look at Gatekeeper. The Gatekeeper project works very similar to Kyverno, except it allows for defining policies in Rego language, which adds complexity but allow for additional customization.