Writing events to pods using client-go

#kubernetes

In the past, I've written controllers utilizing Kubebuilder, where the instantiation of an event recorder is basically done for you with a mgr.GetRecorder() method in the controller manager.

When writing a small binary with client-go, however, I found it surprisingly difficult. It took digging through kubernetes source code to find a couple examples, and I thought I'd document it here.

Scaffolding

Before running the code, let's spin up a pod in the default namespace kubectl run nginx --image=nginx --namespace default

We will be attaching sample events to this pod.

Code

Full code here since some of the packages are specifically named

package main

import (
    "context"
    "path/filepath"
    "time"

    log "github.com/sirupsen/logrus"
    corev1 "k8s.io/api/core/v1"
    v1 "k8s.io/api/core/v1"
    metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
    "k8s.io/apimachinery/pkg/runtime"
    "k8s.io/client-go/kubernetes"
    typedv1core "k8s.io/client-go/kubernetes/typed/core/v1"
    "k8s.io/client-go/rest"
    "k8s.io/client-go/tools/clientcmd"
    "k8s.io/client-go/tools/record"
    "k8s.io/client-go/util/homedir"
)

func main() {
    kubeconfig := filepath.Join(homedir.HomeDir(), ".kube", "config")
    config, err := clientcmd.BuildConfigFromFlags("", kubeconfig)
    if err != nil {
        log.Info("connecting with config in-cluster")
        config, err = rest.InClusterConfig()
        if err != nil {
            log.Fatal(err)
        }
    }
    if err != nil {
        log.Fatal(err)
    }

    clientset, err := kubernetes.NewForConfig(config)
    if err != nil {
        log.Fatal(err)
    }

    pod, err := clientset.CoreV1().Pods("default").Get(context.TODO(), "nginx", metav1.GetOptions{})
    if err != nil {
        log.Fatal(err)
    }

    scheme := runtime.NewScheme()
    _ = corev1.AddToScheme(scheme)

    // all the good events stuff is here
    eventBroadcaster := record.NewBroadcaster()
    eventBroadcaster.StartStructuredLogging(4)
    eventBroadcaster.StartRecordingToSink(&typedv1core.EventSinkImpl{Interface: clientset.CoreV1().Events("")})
    eventRecorder := eventBroadcaster.NewRecorder(scheme, v1.EventSource{})
    eventRecorder.Event(pod, corev1.EventTypeNormal, "is this thing on", "is this thing on")
    eventBroadcaster.Shutdown()
    time.Sleep(2 * time.Second)
}

Things worth noting:

You must add the corev1 scheme for the pod, as it is a corev1 object type. If you are using any other object type, make sure to add the appropriate scheme.
I added a time.Sleep() because of the asynchronous nature of this. Without some wait time, the process will exit too quickly and no events will populate.
eventRecorder.Eventf() is a lot more useful than my above example
If anyone understands the internal workings of k8s events better, please send some articles my way ;)

Seeing the events with kubectl

We can see it by querying events:

(⎈|kind-kind:default)learning/write-events-client-go » k get events -n default | grep "is this thing on"
71s         Normal   is this thing on   pod/nginx   is this thing on

But more interestingly, we can also see them attached to the pod object itself:

(⎈|kind-kind:default)learning/write-events-client-go » k describe pod nginx | grep -A 10 Events
Events:
  Type    Reason            Age   From               Message
  ----    ------            ----  ----               -------
  Normal  Scheduled         37s   default-scheduler  Successfully assigned default/nginx to kind-control-plane
  Normal  Pulling           37s   kubelet            Pulling image "nginx"
  Normal  Pulled            36s   kubelet            Successfully pulled image "nginx" in 1.112899376s
  Normal  Created           36s   kubelet            Created container nginx
  Normal  Started           36s   kubelet            Started container nginx
  Normal  is this thing on  10s                      is this thing on