Building Custom Kubernetes Operators: A Technical Guide

Kubernetes operators are custom-built controllers that extend Kubernetes functionality to automate the deployment, management, and operations of specific applications or services. This guide will walk you through the process of building a custom Kubernetes operator, focusing on the technical aspects and using code examples.

Understanding Kubernetes Operators

Kubernetes operators are a subcategory of controllers that utilize custom resources (CRDs) to manage complex stateful applications. Unlike generic controllers, operators focus on a specific domain and are designed to handle the full lifecycle of applications, including deployment, upgrades, and scaling.

Setting Up the Environment

To build a Kubernetes operator, you need to set up your development environment. Here are the steps to get started:

1. Install the Operator SDK: The Operator SDK is a tool for building, testing, and packaging operators. You can install it using the following command:

   curl -LO https://github.com/operator-framework/operator-sdk/releases/download/v1.25.0/operator-sdk-v1.25.0-x86_64-linux-gnu
   chmod +x operator-sdk-v1.25.0-x86_64-linux-gnu
   sudo mv operator-sdk-v1.25.0-x86_64-linux-gnu /usr/local/bin/operator-sdk

1. Initialize the Operator Project: Create a new directory for your operator project and initialize it using the Operator SDK:

   mkdir -p $GOPATH/src/operators && cd $GOPATH/src/operators
   operator-sdk init --domain example.com --repo github.com/example-inc/my-operator

1. Create APIs and Custom Resources: Define the APIs and custom resources for your operator. For example, let's create an API for a Traveller resource:

   operator-sdk create api --version=v1alpha1 --kind=Traveller

This command will create the necessary files, including the Traveller CRD and the controller to manage it.

Defining the Custom Resource Definition (CRD)

The CRD defines the structure of the custom resource. Here is an example of a Traveller CRD:

apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
  name: travellers.example.com
spec:
  group: example.com
  versions:
    - name: v1alpha1
      served: true
      storage: true
      schema:
        openAPIV3Schema:
          type: object
          properties:
            spec:
              type: object
              properties:
                name:
                  type: string
                destination:
                  type: string
  scope: Namespaced
  names:
    plural: travellers
    singular: traveller
    kind: Traveller
    shortNames:
      - trv

Writing the Controller

The controller is responsible for reconciling the desired state of the custom resource with the actual state in the cluster. Here is an example of a basic Traveller controller:

package controllers

import (
    "context"
    "fmt"
    "log"

    "k8s.io/apimachinery/pkg/runtime"
    ctrl "sigs.k8s.io/controller-runtime"
    "sigs.k8s.io/controller-runtime/pkg/client"
    "sigs.k8s.io/controller-runtime/pkg/controller"
    "sigs.k8s.io/controller-runtime/pkg/controller/controllerutil"

    examplev1alpha1 "github.com/example-inc/my-operator/api/v1alpha1"
)

// TravellerReconciler reconciles a Traveller object
type TravellerReconciler struct {
    client.Client
    Scheme *runtime.Scheme
}

//+kubebuilder:rbac:groups=example.com,resources=travellers,verbs=get;list;watch;create;update;patch;delete
//+kubebuilder:rbac:groups=example.com,resources=travellers/status,verbs=get;update;patch
//+kubebuilder:rbac:groups=example.com,resources=travellers/finalizers,verbs=update

func (r *TravellerReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {
    log.Println("Reconciling Traveller")

    traveller := &examplev1alpha1.Traveller{}
    err := r.Get(ctx, req.NamespacedName, traveller)
    if err != nil {
        log.Println(err)
        return ctrl.Result{}, err
    }

    // Example logic to reconcile the Traveller resource
    if traveller.Spec.Name == "" {
        traveller.Spec.Name = "Default Name"
        err = r.Update(ctx, traveller)
        if err != nil {
            log.Println(err)
            return ctrl.Result{}, err
        }
    }

    return ctrl.Result{}, nil
}

func (r *TravellerReconciler) SetupWithManager(mgr ctrl.Manager) error {
    return ctrl.NewControllerManagedBy(mgr).
        For(&examplev1alpha1.Traveller{}).
        Complete(r)
}

Running the Operator

To run the operator, you need to build and deploy it. Here are the steps:

1. Build the Operator:

   make build

1. Deploy the Operator:

   make deploy

1. Verify the Operator:

   kubectl get deployments -n my-operator-system

Testing the Operator

Testing is crucial to ensure the operator works as expected. Here is an example of how to test the Traveller operator:

1. Create a Sample Resource:

   apiVersion: example.com/v1alpha1
   kind: Traveller
   metadata:
     name: example-traveller
   spec:
     name: John Doe
     destination: Paris

1. Apply the Resource:

   kubectl apply -f sample-traveller.yaml

1. Verify the Resource:

   kubectl get traveller example-traveller -o yaml

Conclusion

Building custom Kubernetes operators involves defining custom resources, writing controllers to manage these resources, and deploying the operator to a Kubernetes cluster. This guide has walked you through the technical steps required to build a basic operator, focusing on the code and configuration necessary for a functional operator. In the context of Platform Engineering, such custom operators can significantly enhance the automation and management capabilities of Kubernetes, making it easier to deploy and manage complex stateful applications.

Additional Resources

• Kubebuilder: A framework for building Kubernetes APIs and operators.
• Operator SDK: A tool for building, testing, and packaging operators.
• Custom Resource Definitions: Detailed documentation on defining CRDs.

By following these steps and using the provided code examples, you can create custom Kubernetes operators tailored to your specific application needs.