Accessing devices from inside a Docker container without using the --privileged flag

Accessing devices from inside a docker container is a bit tricky.

The most common suggestion is to use the --privileged flag while starting a docker container. But from a security perspective, that is a terrible thing to do. Using the --privileged flag would give the container access and all capabilities to all the devices connected to the host (i.e. everything under the /dev directory). This article from Trend Micro explains the risks in much more detail. You could use this site as a reference for linux capabilities.

While looking for a more secure solution, I came across this post by Marc Merlin. I used Marc's post as a reference to solve my problem.

The problem

Connect a USB device to your PC. Run the command lsusb in your terminal. Find your USB device from the output list and note down the bus number and device number. You could use the bus number and device number to locate your device inside the /dev directory at /dev/bus/usb/<bus_number>/<device_number>.

This guide works for other kinds of devices as well like a block storage device, or a loop device, or an audio device. You just need to know where the device is located in your file system.

Now, start a container and try to locate the USB device inside the /dev directory inside the container. You will notice that it's missing.

sudo docker run \
--rm -it \
ubuntu:20.04

ls /dev

The goal

The goal is to access the connected device from inside the container and be able to use it.

The naive solution: using the --privileged flag

The most common solution is to start a container with the --privileged flag.

sudo docker run \
--rm -it \
--privileged \
ubuntu:20.04

ls /dev

But as mentioned earlier, this should be avoided.

Using the --device flag

The --device exposes devices to a container. You could mount either a single device like /dev/bus/usb/<bus_number>/<device_number> or /dev/ttyUSBxyz, or a directory of devices like /dev/bus/usb.

sudo docker run \
--rm -it \
--device /dev/bus/usb \
ubuntu:20.04

ls /dev

This method works fine. But it gives you access only to the devices that were connected to the host while starting the continer. If you reconnect a device, or connect a new device, then you won't be able to access it from inside the container.

Using the --device-cgroup-rule flag

The --device-cgroup-rule flag allows you add a more permissive rule to a container allowing it access to a wider range of devices. In a way, this flag also allows you to limit access only to some devices.

Before creating the container, you need to know the major and minor number of the device you want the container to have access to. You can find the major and minor number of a device by running the ls -l command on your device or device directory.

Here the major number is 189. And the number after 189 is the minor number for the corresponding device.

ls -l /dev/bus/usb/001/
total 0
crw-rw-r--  1 root root  189, 0 Dec 13 18:09 001
crw-rw-r--  1 root root  189, 1 Dec 13 18:09 002
crw-rw-r--  1 root root  189, 2 Dec 13 18:09 003
crw-rw-r--  1 root root  189, 3 Dec 13 18:09 004
crw-rw-r--  1 root root  189, 4 Dec 13 18:09 005
crw-rw-r--  1 root root  189, 5 Dec 13 18:09 006
crw-rw----+ 1 root audio 189, 6 Dec 13 19:48 007

The device-cgroup-rule is written in the following format:

type major:minor mode

type: a (all), or b (block), or c (char)
major and minor: either a number, or * for all
mode: a composition of r (read), w (write), and m (mknod)

Now start a container with the --device-cgroup-rule flag. This gives the container access to all the devices with major number 189.

sudo docker run \
--rm -it \
--device /dev/bus/usb \
--device-cgroup-rule 'a 189:* rwm' \
ubuntu:20.04

ls /dev

If you want the container to have access to all the devices, you could use --device-cgroup-rule 'a *:* rwm'.

So everything should work fine now, right? Not really. If you try to reconnecting a device, or connect a new device, you still won't be able to access it from inside the container. You need to run the mknod command inside the container each time there is a device change. That's too much work. There is a better way to solve this problem.

Mounting the devices as a volume

Instead of using the --device flag, mount the devices or device directory as a volume.

sudo docker run \
--rm -it \
-v /dev/bus/usb:/dev/bus/usb \
--device-cgroup-rule 'a 189:* rwm' \
ubuntu:20.04

ls /dev

Now it would work perfectly fine with both old and new devices, as long as the device has a major number 189.

Adding capabilities to the docker container

Note: this step is completely optional.

In some cases, you may actually want the docker container to have some capabilities. For this, you could use the --cap-add flag. You could use this site as a reference for linux capabilities.

sudo docker run \
--rm -it \
-v /dev/bus/usb:/dev/bus/usb \
--device-cgroup-rule 'a 189:* rwm' \
--cap-add SYS_PTRACE \
ubuntu:20.04

ls /dev

Conclusion

Mounting the devices as a volume along with using the --device-cgroup-rule flag is a good way to avoid creating privileged containers.