Ephemeral Containers

In this blog I want to talk about ephemeral containers.

Ephemeral containers: a special type of container that runs temporarily in an existing Pod to accomplish user-initiated actions such as troubleshooting. You use ephemeral containers to inspect services rather than to build applications.

Ephemeral containers are useful for debugging and for interactive troubleshooting when kubectl exec is insufficient because a container has crashed or a container image doesn't include debugging utilities.

Ephemeral containers share process namespaces and resources with the pod. They are especially useful for lightweight distroless containers.

Ephemeral container:

• live until exit
• never auto restarts
• does not have ports, livenessProbe, readinessProbe
• immutable - resources setting not allowed

Deploy PostgreSQL in Kubernetes:

See my blog

Make sure the PostgreSQL pod is running:

kubectl get pods
NAME                        READY   STATUS    RESTARTS       AGE
postgres-5c7d8b656d-xj5b2   1/1     Running   1 (7m2s ago)   45h

Get container name(s), image for each pod in the k8s cluster:

kubectl get pods -o=custom-columns=PodName:.metadata.name,Containers:.spec.containers[*].name,Image:.spec.containers[*].image

PodName                     Containers                  Image
postgres-5c7d8b656d-xj5b2   postgres                    postgres:latest

Here is example how to run ephemeral container with a busybox shell to debug our pod postgres-5c7d8b656d-xj5b

kubectl debug -it postgres-5c7d8b656d-xj5b2 --image=busybox --target=postgres

This command adds a new busybox container and attaches to it. The --target parameter targets the process namespace of another container.

Now we can see processes from the “original” pod that are sharing the same namespace.

/ # ps aux
PID   USER     TIME  COMMAND
    1 999       0:00 postgres
   26 999       0:00 postgres: checkpointer
   27 999       0:00 postgres: background writer
   29 999       0:00 postgres: walwriter
   30 999       0:00 postgres: autovacuum launcher
   31 999       0:00 postgres: logical replication launcher
   46 root      0:00 sh
   52 root      0:00 ps aux
   …

The ephemeral container is running in the same network context as the “original” container. It allows us to inspect what is going on inside of this pod from its network perspective, run ping, etc.

/ # ping 8.8.8.8
PING 8.8.8.8 (8.8.8.8): 56 data bytes
64 bytes from 8.8.8.8: seq=0 ttl=114 time=16.472 ms
64 bytes from 8.8.8.8: seq=1 ttl=114 time=16.484 ms
…

Here is example how to run ephemeral container with a ubuntu injected to debug our pod postgres-5c7d8b656d-xj5b

kubectl debug -it postgres-5c7d8b656d-xj5b2 --image=ubuntu --target=postgres

apt-get update

apt-get install -y tcpdump