Kubernetes add-on compliance with Projectsveltos and Lua

Add-ons extend the functionality of Kubernetes. Projectsveltos is a Kubernetes add-on controller that can be used to automate the deployment and management of add-ons across multiple Kubernetes clusters.

Kubernetes add-on compliance

Kubernetes add-on compliance refers to the process of ensuring that all Kubernetes add-ons within a cluster meet the specific security and compliance requirements of an organization. This compliance helps maintain consistency, compatibility, and interoperability between different Kubernetes environments. It ensures that add-ons are well-behaved, reliable, and can seamlessly integrate with the Kubernetes ecosystem.

Add-on compliance rules using Lua

With Projectsveltos, it is easy to define constraints and specify their enforcement. If a constraint rule is defined for a Kubernetes cluster, Projectsveltos will always validate add-ons against existing compliance rules before deploying them in the cluster. Only Kubernetes add-ons that comply with the existing rules will be deployed.

We have previously discussed how to enforce compliance rules using Projectsveltos and OpenAPI. Projectsveltos also supports defining compliance rules using Lua. Lua is more flexible than OpenAPI because it allows resources to be validated both independently and together. For example, consider an Helm chart, which contains multiple Kubernetes resources. Using Projectsveltos and Lua, we can define a compliance rule that requires each deployment in the namespace foo to have an associated HorizontalPodAutoscaler.

Here is all the needed configuration:

apiVersion: lib.projectsveltos.io/v1alpha1
kind: AddonCompliance
metadata:
 name: depl-replica
spec:
  clusterSelector: env=production
  luaValidationRefs:
  - namespace: default
    name: depl-horizontalpodautoscaler
    kind: ConfigMap

apiVersion: v1
data:
  lua.yaml: |
    function evaluate()
        local hs = {}
        hs.valid = true
        hs.message = ""

        local deployments = {}
        local autoscalers = {}

        -- Separate deployments and services from the resources
        for _, resource in ipairs(resources) do
            local kind = resource.kind
            if resource.metadata.namespace == "foo" then
                if kind == "Deployment" then
                    table.insert(deployments, resource)
                elseif kind == "HorizontalPodAutoscaler" then
                    table.insert(autoscalers, resource)
                end
            end
        end

        -- Check for each deployment if there is a matching HorizontalPodAutoscaler
        for _, deployment in ipairs(deployments) do
            local deploymentName = deployment.metadata.name
            local matchingAutoscaler = false

            for _, autoscaler in ipairs(autoscalers) do
                if autoscaler.spec.scaleTargetRef.name == deployment.metadata.name then
                    matchingAutoscaler = true
                    break
                end
            end

            if not matchingAutoscaler then
                hs.valid = false
                hs.message = "No matching autoscaler found for deployment: " .. deploymentName
                break
            end
        end

        return hs
    end
kind: ConfigMap
metadata:
  name: depl-horizontalpodautoscaler
  namespace: default

Validating your Lua policies

If you want to validate your Lua policies:

1. clone sveltos addon-controller repo:
   git clone git@github.com:projectsveltos/addon-controller.git

2. cd controllers/validate_lua

3. Create your own directory within the validate_lua directory. Inside this directory, create the following files:

   • lua_policy.yaml: This file should contain your Lua policy;
   • valid_resource.yaml: This file should contain the resources that satisfy the Lua policy;
   • invalid_resource.yaml: This file should contain the resources that do not satisfy the Lua policy;

4. run make test from repo directory.

Running make test will initiate the validation process, which thoroughly tests your Lua policies against the provided resource files. This procedure ensures that your defined policy is not only syntactically correct but also functionally accurate. By executing the validation tests, you can gain confidence in the correctness and reliability of your Lua policies. By following these steps, you can easily validate your Lua policies using the Projectsveltos addon-controller repository.

Choosing this Approach over Using an Admission Controller

Let’s explore the advantages of choosing the approach provided by Sveltos for add-on deployment and constraint enforcement over relying on an admission controller like Kyverno or OPA:

1. Synchronization without Hassle: When using an admission controller, it is crucial to ensure that no add-ons are deployed until the controller is up and running. This often requires implementing synchronization mechanisms to coordinate the deployment process. With Sveltos, this is taken care of for you. The add-on controller in Sveltos patiently waits for the add-on constraint controller to load all existing constraints specific to each cluster. This guarantees a smooth and orderly deployment process without the need for additional synchronization mechanisms.
2. Consistency in Resource Deployment: When deploying resources using an Helm chart or similar mechanisms, multiple resources are typically deployed together as part of a cohesive unit. With the approach offered by Sveltos, a strict rule applies: either all the resources in the deployment satisfy the existing constraints and are valid, or none of them are deployed. This ensures consistency and prevents partial or incomplete deployments, leading to a more reliable and predictable deployment process.

👏 Support this project

If you enjoyed this article, please check out the GitHub repo for the project. You can also star 🌟 the project if you found it helpful.

The GitHub repo is a great resource for getting started with the project. It contains the code, documentation, and examples. You can also find the latest news and updates on the project on the GitHub repo.

Thank you for reading!