DEV Community

Adam Gardner
Adam Gardner

Posted on

Automatic OpenTelemetry for Kubernetes Jobs (Hands on)

Use the tracepusher Kubernetes Operator to automatically generate OpenTelemetry traces for Kubernetes Job and CronJob resources.

Step 0: Create a cluster & Start Jaeger

If you don't already have a cluster, create one now:

kind create cluster

# Or install jaeger on Kubernetes with the jaeger operator
# https://www.jaegertracing.io/docs/1.53/operator/
docker run --rm --name jaeger \
  -e COLLECTOR_ZIPKIN_HOST_PORT=:9411 \
  -p 6831:6831/udp \
  -p 6832:6832/udp \
  -p 5778:5778 \
  -p 16686:16686 \
  -p 4317:4317 \
  -p 4318:4318 \
  -p 14250:14250 \
  -p 14268:14268 \
  -p 14269:14269 \
  -p 9411:9411 \
  jaegertracing/all-in-one:1.53
Enter fullscreen mode Exit fullscreen mode

Step 1: Install tracepusher Custom Resource Definitions on cluster

kubectl apply -f https://raw.githubusercontent.com/agardnerIT/tracepusher/main/operator/crds.yml
Enter fullscreen mode Exit fullscreen mode

Step 2: Create a JobTracer

Modify the OpenTelemetry collector URL as appropriate for your setup and then apply the file. One JobTracer per namespace please!

This file instructs tracepusher to monitor for all Job resources in the given namespace (you can ignore on a job-by-job basis with annotations that you'll see next):

cat <<EOF | kubectl create -f -
---
apiVersion: tracers.tracepusher.github.io/v1
kind: JobTracer
metadata:
  name: tracer
  namespace: default
spec:
  # Specify your OpenTelemetry collector endpoint
  # Here I've given the docker URI
  collectorEndpoint: "http://host.docker.internal:4318"
EOF
Enter fullscreen mode Exit fullscreen mode

Step 3: Create a Normal k8s Job / CronJob

Create a Job / CronJob as normal. The tracepusher annotations are optional and will override the behaviour of the above JobTracer

cat <<EOF | kubectl create -f -
---
apiVersion: batch/v1
kind: Job
metadata:
  name: pi
  namespace: default
  #annotations:
  #  tracepusher/ignore: "true"
  #  tracepusher/collector: "http://example.com:4318"
spec:
  template:
    spec:
      containers:
      - name: pi
        image: perl:5.34.0
        command: ["perl",  "-Mbignum=bpi", "-wle", "print bpi(2000)"]
      restartPolicy: Never
  backoffLimit: 0
EOF
Enter fullscreen mode Exit fullscreen mode

tracepusher job tracer result

Top comments (0)