Kubectl Exec: Everything You Need to Know

By Eric Goebelbecker

Containers make application portability and delivery easier. They use resources more efficiently than bare metal or virtual machines. They're more portable, lighter, and smaller, making them easier to deploy anywhere, regardless of which cloud provider or on-premises center you need to run your code.

Kubernetes (k8s) supercharges container infrastructure with built-in scaling, reliability, and even more efficient resource usage. But, k8s adds an extra layer of abstraction, too. Many administrative tasks don't work exactly the way you'd expect. Sometimes you find it difficult to pierce the extra layers of abstraction and see what's going on inside your cluster.

In this post, we're going to look at kubectl exec. We'll see how you can use this tool to peek inside Kubernetes containers. We'll look at what exec has to offer and some examples of how to and how not to use it to manage your Kubernetes clusters.

If you want to follow the examples in this article, you'll need a system running Docker with Minikube installed and started.

What is Kubectl Exec?

Kubectl

Kubectl is a command line tool for communicating withKubernetes clusters via the Kubernetes API. You can use it to monitor Kubernetes status, apply manifest files, edit resources, and much more. It's a general admin tool for k8s clusters.

Depending on your operating system, you may need to install kubectl separately. The packages for Docker and Kubernetes may install it for you.

Kubectl Exec

Exec is one of kubectl's most useful tools. You can use it to execute a command inside a container. If you're familiar with Docker, kubectl's exec may remind you of Docker's exec command.

Let's start with an example.

Kubectl and Namespaces

Before we can execute a command in a container, we need to know its name, and before we can list containers, we need to know their namespaces. So, let's start by viewing the namespaces inside our cluster.

First, start with kubectl get namespace:

% kubectl get namespace
NAME              STATUS   AGE
default           Active   2d23h
kube-node-lease   Active   2d23h
kube-public       Active   2d23h
kube-system       Active   2d23h

We have four namespaces: default, kube-node-lease, kube-public, and kube-system.

Next, we'll retrieve a list of containers with get pods and a namespace by adding the --namespace command line argument.

 % kubectl --namespace=default get pods
No resources found in default namespace.

I haven't run anything on my Minikube cluster yet, so the default namespace is empty. Let's look at kube-system:

% kubectl --namespace=kube-system get pods
NAME                               READY   STATUS    RESTARTS        AGE
coredns-6d4b75cb6d-xsqnx           1/1     Running   0               3d
etcd-minikube                      1/1     Running   0               3d
kube-apiserver-minikube            1/1     Running   0               3d
kube-controller-manager-minikube   1/1     Running   0               3d
kube-proxy-hqxbp                   1/1     Running   0               3d
kube-scheduler-minikube            1/1     Running   0               3d
storage-provisioner                1/1     Running   287 (46h ago)   3d

There are a few containers in this namespace. So, let's look at them.

Exec a Shell in a Kubernetes Container

Now we can start a shell in one of the Kubernetes containers. Let's use kube-proxy-hqxbp:

% kubectl exec --namespace=kube-system kube-proxy-hqxbp -it -- sh
# ls
bin  boot  dev  etc  go-runner  home  lib  media  mnt  opt  proc  root  run  sbin  srv  sys  tmp  usr  var
# pwd
/

Kubectl started a Bourne shell inside the container and redirected the input and output back to my shell. Next, I listed the directory sh put me in, and then I used pwd to display its name.

Let's break down this command line next.

• exec is the subcommand we want to run.
• --name=kube-system tells kubectl which namespace the container is running in.
• kube-proxy-hqxbp is the container.
• -it tells exec to redirect the shell's input and output streams back to the controlling shell. This should look familiar if you've used Docker's exec command. The I tells exec to direct standard input to your shell. The t tells exec that the input is coming from a tty.
• The next set of arguments is two dashes (--) followed by the command you want exec to run. In this case, it's sh, the Bourne shell. The two dashes separate the command's arguments from kubectl's.

Kubectl Exec Examples

Now that we understand how to run exec, we can look at a few examples.

We didn't need to start and quit a shell to retrieve the contents of the root directory. We can pass the ls command to sh and have exec return the results to us.

% kubectl exec --namespace=kube-system kube-proxy-hqxbp -it -- sh -c ls
bin   dev  go-runner  lib    mnt  proc  run   srv  tmp  var

Actually, this isn't an interactive command, so we don't need -it. Let's try it without it.

% kubectl exec --namespace=kube-system kube-proxy-hqxbp -- sh -c "ls"
bin
boot
dev
etc
go-runner
home
lib
media
mnt
opt
proc
root
run
sbin
srv
sys
tmp
usr
var

That's interesting! As a result of leaving out the -it ls' behavior changed. It doesn't use more than one column for a listing of the output isn't to a terminal. This makes parsing the results much easier in shell scripts.

Let's look in the /bin directory and see what else we can do in this container.

% kubectl exec --namespace=kube-system kube-proxy-hqxbp -- sh -c "ls -l /bin"
total 3384
-rwxr-xr-x 1 root root  39832 Sep 22  2020 cat
-rwxr-xr-x 1 root root  64512 Sep 22  2020 chgrp
-rwxr-xr-x 1 root root  60368 Sep 22  2020 chmod
-rwxr-xr-x 1 root root  64528 Sep 22  2020 chown
-rwxr-xr-x 1 root root 138896 Sep 22  2020 cp
-rwxr-xr-x 1 root root 129544 Dec 10  2020 dash
-rwxr-xr-x 1 root root 101384 Sep 22  2020 date
-rwxr-xr-x 1 root root  80984 Sep 22  2020 dd
-rwxr-xr-x 1 root root  89824 Sep 22  2020 df
-rwxr-xr-x 1 root root 143088 Sep 22  2020 dir
lrwxrwxrwx 1 root root      8 Nov  7  2019 dnsdomainname -> hostname
lrwxrwxrwx 1 root root      8 Nov  7  2019 domainname -> hostname
-rwxr-xr-x 1 root root  35632 Sep 22  2020 echo
-rwxr-xr-x 1 root root     28 Nov  9  2020 egrep
(trimmed)

This container has a limited set of command line tools installed, so there's not much we can do with it. Let's look at a better container.

Kubectl Exec With a User Container

First, start an Ubuntu container in the default namespace. The official container isn't bundled with a default command, so run it with an interactive shell:

% kubectl run --namespace=default -it exec-test-ubuntu --image=ubuntu bash
If you don't see a command prompt, try pressing enter.
root@exec-test-ubuntu:/# ps -ef
UID          PID    PPID  C STIME TTY          TIME CMD
root           1       0  0 18:39 pts/0    00:00:00 bash
root          11       1  0 18:39 pts/0    00:00:00 ps -ef

Note that you have to pass in the -it argument this time, or bash will exit. This puts us into a shell. The command prompt matches the container name. This container has the ps command, so we can see what's running inside the container. It shows the bash process we're interacting with and the ps command itself.

Next, in another window, exec a shell into the same container.

 % kubectl exec -it exec-test-ubuntu  -- bash
root@exec-test-ubuntu:/#

Now, go back to the first command prompt and run a process listing:

root@exec-test-ubuntu:/# ps -ef
UID          PID    PPID  C STIME TTY          TIME CMD
root           1       0  0 18:39 pts/0    00:00:00 bash
root          12       0  0 18:39 pts/1    00:00:00 bash
root          21       1  0 18:39 pts/0    00:00:00 ps -ef

There are two bash sessions now! So, it's clear we have two shells into the same container and an idea of how exec works.

Kubectl Exec Notes

As we've seen from the examples above, exec gives you the ability to do anything to a container, but that doesn't mean you should. Exec is a tool for inspecting containers, not for running applications or changing container configuration.

If you need to update a container's configuration or code, build and deploy a new image. Exec-ing into the container and installing new packages or copying in new files will cause reproducibility issues and violate the spirit of containerization.

Kubectl Exec for Container Inspection

We've examined kubectl's exec function and how it works. We saw how to figure out which namespace your containers are running in. Then we used exec to execute a program inside the desired container. We saw how you can use kubectl exec to run both interactive shells and commands that simply return results. Next, we started a container in the default namespace and used exec to inspect it.

Kubectl exec is a powerful tool for viewing the status and contents of containers in your Kubernetes clusters. Now that you're familiar with how it works, you can use it to keep your k8s clusters running smoothly!  

This post was written by Eric Goebelbecker. Eric has worked in the financial markets in New York City for 25 years, developing infrastructure for market data and financial information exchange (FIX) protocol networks. He loves to talk about what makes teams effective (or not so effective!).