Containers make application portability and delivery easier. They use resources more efficiently than bare metal or virtual machines. They're more portable, lighter, and smaller, making them easier to deploy anywhere, regardless of which cloud provider or on-premises center you need to run your code.
Kubernetes (k8s) supercharges container infrastructure with built-in scaling, reliability, and even more efficient resource usage. But, k8s adds an extra layer of abstraction, too. Many administrative tasks don't work exactly the way you'd expect. Sometimes you find it difficult to pierce the extra layers of abstraction and see what's going on inside your cluster.
In this post, we're going to look at kubectl exec. We'll see how you can use this tool to peek inside Kubernetes containers. We'll look at what exec has to offer and some examples of how to and how not to use it to manage your Kubernetes clusters.
Kubectl is a command line tool for communicating withKubernetes clusters via the Kubernetes API. You can use it to monitor Kubernetes status, apply manifest files, edit resources, and much more. It's a general admin tool for k8s clusters.
Depending on your operating system, you may need to install kubectl separately. The packages for Docker and Kubernetes may install it for you.
Exec is one of kubectl's most useful tools. You can use it to execute a command inside a container. If you're familiar with Docker, kubectl's exec may remind you of Docker's exec command.
Let's start with an example.
Before we can execute a command in a container, we need to know its name, and before we can list containers, we need to know their namespaces. So, let's start by viewing the namespaces inside our cluster.
First, start with kubectl get namespace:
% kubectl get namespace NAME STATUS AGE default Active 2d23h kube-node-lease Active 2d23h kube-public Active 2d23h kube-system Active 2d23h
We have four namespaces: default, kube-node-lease, kube-public, and kube-system.
Next, we'll retrieve a list of containers with get pods and a namespace by adding the --namespace command line argument.
% kubectl --namespace=default get pods No resources found in default namespace.
I haven't run anything on my Minikube cluster yet, so the default namespace is empty. Let's look at kube-system:
% kubectl --namespace=kube-system get pods NAME READY STATUS RESTARTS AGE coredns-6d4b75cb6d-xsqnx 1/1 Running 0 3d etcd-minikube 1/1 Running 0 3d kube-apiserver-minikube 1/1 Running 0 3d kube-controller-manager-minikube 1/1 Running 0 3d kube-proxy-hqxbp 1/1 Running 0 3d kube-scheduler-minikube 1/1 Running 0 3d storage-provisioner 1/1 Running 287 (46h ago) 3d
There are a few containers in this namespace. So, let's look at them.
Now we can start a shell in one of the Kubernetes containers. Let's use kube-proxy-hqxbp:
% kubectl exec --namespace=kube-system kube-proxy-hqxbp -it -- sh # ls bin boot dev etc go-runner home lib media mnt opt proc root run sbin srv sys tmp usr var # pwd /
Kubectl started a Bourne shell inside the container and redirected the input and output back to my shell. Next, I listed the directory sh put me in, and then I used pwd to display its name.
Let's break down this command line next.
- exec is the subcommand we want to run.
- --name=kube-system tells kubectl which namespace the container is running in.
- kube-proxy-hqxbp is the container.
- -it tells exec to redirect the shell's input and output streams back to the controlling shell. This should look familiar if you've used Docker's exec command. The I tells exec to direct standard input to your shell. The t tells exec that the input is coming from a tty.
- The next set of arguments is two dashes (--) followed by the command you want exec to run. In this case, it's sh, the Bourne shell. The two dashes separate the command's arguments from kubectl's.
Now that we understand how to run exec, we can look at a few examples.
We didn't need to start and quit a shell to retrieve the contents of the root directory. We can pass the ls command to sh and have exec return the results to us.
% kubectl exec --namespace=kube-system kube-proxy-hqxbp -it -- sh -c ls bin dev go-runner lib mnt proc run srv tmp var
Actually, this isn't an interactive command, so we don't need -it. Let's try it without it.
% kubectl exec --namespace=kube-system kube-proxy-hqxbp -- sh -c "ls" bin boot dev etc go-runner home lib media mnt opt proc root run sbin srv sys tmp usr var
That's interesting! As a result of leaving out the -it ls' behavior changed. It doesn't use more than one column for a listing of the output isn't to a terminal. This makes parsing the results much easier in shell scripts.
Let's look in the /bin directory and see what else we can do in this container.
% kubectl exec --namespace=kube-system kube-proxy-hqxbp -- sh -c "ls -l /bin" total 3384 -rwxr-xr-x 1 root root 39832 Sep 22 2020 cat -rwxr-xr-x 1 root root 64512 Sep 22 2020 chgrp -rwxr-xr-x 1 root root 60368 Sep 22 2020 chmod -rwxr-xr-x 1 root root 64528 Sep 22 2020 chown -rwxr-xr-x 1 root root 138896 Sep 22 2020 cp -rwxr-xr-x 1 root root 129544 Dec 10 2020 dash -rwxr-xr-x 1 root root 101384 Sep 22 2020 date -rwxr-xr-x 1 root root 80984 Sep 22 2020 dd -rwxr-xr-x 1 root root 89824 Sep 22 2020 df -rwxr-xr-x 1 root root 143088 Sep 22 2020 dir lrwxrwxrwx 1 root root 8 Nov 7 2019 dnsdomainname -> hostname lrwxrwxrwx 1 root root 8 Nov 7 2019 domainname -> hostname -rwxr-xr-x 1 root root 35632 Sep 22 2020 echo -rwxr-xr-x 1 root root 28 Nov 9 2020 egrep (trimmed)
This container has a limited set of command line tools installed, so there's not much we can do with it. Let's look at a better container.
First, start an Ubuntu container in the default namespace. The official container isn't bundled with a default command, so run it with an interactive shell:
% kubectl run --namespace=default -it exec-test-ubuntu --image=ubuntu bash If you don't see a command prompt, try pressing enter. root@exec-test-ubuntu:/# ps -ef UID PID PPID C STIME TTY TIME CMD root 1 0 0 18:39 pts/0 00:00:00 bash root 11 1 0 18:39 pts/0 00:00:00 ps -ef
Note that you have to pass in the -it argument this time, or bash will exit. This puts us into a shell. The command prompt matches the container name. This container has the ps command, so we can see what's running inside the container. It shows the bash process we're interacting with and the ps command itself.
Next, in another window, exec a shell into the same container.
% kubectl exec -it exec-test-ubuntu -- bash root@exec-test-ubuntu:/#
Now, go back to the first command prompt and run a process listing:
root@exec-test-ubuntu:/# ps -ef UID PID PPID C STIME TTY TIME CMD root 1 0 0 18:39 pts/0 00:00:00 bash root 12 0 0 18:39 pts/1 00:00:00 bash root 21 1 0 18:39 pts/0 00:00:00 ps -ef
There are two bash sessions now! So, it's clear we have two shells into the same container and an idea of how exec works.
As we've seen from the examples above, exec gives you the ability to do anything to a container, but that doesn't mean you should. Exec is a tool for inspecting containers, not for running applications or changing container configuration.
If you need to update a container's configuration or code, build and deploy a new image. Exec-ing into the container and installing new packages or copying in new files will cause reproducibility issues and violate the spirit of containerization.
We've examined kubectl's exec function and how it works. We saw how to figure out which namespace your containers are running in. Then we used exec to execute a program inside the desired container. We saw how you can use kubectl exec to run both interactive shells and commands that simply return results. Next, we started a container in the default namespace and used exec to inspect it.
Kubectl exec is a powerful tool for viewing the status and contents of containers in your Kubernetes clusters. Now that you're familiar with how it works, you can use it to keep your k8s clusters running smoothly!
This post was written by Eric Goebelbecker. Eric has worked in the financial markets in New York City for 25 years, developing infrastructure for market data and financial information exchange (FIX) protocol networks. He loves to talk about what makes teams effective (or not so effective!).