DEV Community

Charles Uneze
Charles Uneze

Posted on • Updated on

Docker ARP Analysis Guide

#docker #networking #ubuntu #multipass

Table of Contents

Read this as technical documentation instead.

Introduction

I've been immersing myself in CI/CD pipeline studies lately, and now that I've gotten a good grasp of it, I can finally dedicate my full attention to understanding container technologies. To start, I decided to focus on networking, which is the stack I'm most familiar with.
In this simple how-to guide, I'll walk you through observing the Address Resolution Protocol (ARP) in action within a Docker environment.
An Address Resolution Protocol(ARP) allows a container to learn the MAC address of another device (in this lab, it’s the bridge and other containers) dynamically.
Without an ARP, a ping between containers, external networks, or even a web request between containers and other devices fails.

Requirements

I recently made the switch to a Mac and found using Multipass from Canonical simpler than spinning up a VM with tools like Virtualbox. However, when it comes to container networking labs, non-Linux systems are not recommended. While I downloaded Docker Desktop for Mac, I had trouble seeing all the Docker network interfaces.
For Windows users, I will advise you to use WSL2 instead, it’s easier to deploy and manage compared to having to use a virtual machine like Virtualbox.
Keep in mind that the Ubuntu installation requirement is only for Mac and Linux users. This guide will use three terminal tabs throughout.

Install Homebrew and Multipass

Download Homebrew

# Terminal-1 Mac/Linux
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
Enter fullscreen mode Exit fullscreen mode

Download Multipass

# Terminal-1 Mac/Linux
brew install multipass
Enter fullscreen mode Exit fullscreen mode

Download an Ubuntu Server from Multipass

This Ubuntu server will be customized to run the same specifications as an AWS EC2-T2 Micro instance type.

  • 1 CPU
  • 1 GB of RAM
  • 8 GB of disk
  • version 22.04

Simple and fast, right?

# Terminal-1 Mac/Linux
multipass launch jammy --name=ubuntu --cpus=1 --disk=8G --memory=1G
Enter fullscreen mode Exit fullscreen mode

jammy is the image name for Ubuntu server version 22.04.

Go to the Shell of the Ubuntu Server 

# Terminal-1 Mac/Linux
multipass shell ubuntu
Enter fullscreen mode Exit fullscreen mode

Install Docker

Download docker on the Ubuntu server.

# Terminal-1 Ubuntu
sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin

# Close this Ubuntu shell.
# Sometimes you might need to use the exit command severally
# to successfully exit the shell.
exit
Enter fullscreen mode Exit fullscreen mode

Mount a Local Folder to an Ubuntu Directory

Mount the Documents/ folder in your Mac, or other machine to the mnt/ directory in the Ubuntu server running inside Multipass.

# Terminal-1 Mac/Linux
multipass mount /Users/apple/Documents /mnt/
Enter fullscreen mode Exit fullscreen mode

Verify Ubuntu Server Installation 

apple@Charles-MBP ~ % multipass info ubuntu
Name:           ubuntu
State:          Running
Snapshots:      0
IPv4:           192.168.64.4
                172.17.0.1
Release:        Ubuntu 22.04.3 LTS
Image hash:     9dxa2awl28c8 (Ubuntu 22.04 LTS)
CPU(s):         1
Load:           0.00 0.00 0.00
Disk usage:     2.3GiB out of 7.7GiB
Memory usage:   201.4MiB out of 951.6MiB
Mounts:         /Users/apple/Documents => /mnt
                    UID map: 501:default
                    GID map: 20:default
Enter fullscreen mode Exit fullscreen mode

Install Wireshark

Choose your preferred machine and download.

Architecture Deployment Guide

I have a simple architecture that deploys two docker containers in two different subnets. In docker, you can attach one container to several subnets. This is achieved by a new interface being created and assigned to that subnet.

architecture

table

Enter Into the Shell of the Ubuntu Server Again 

# Terminal-1 Mac/Linux
multipass shell ubuntu
Enter fullscreen mode Exit fullscreen mode

Create a New Network

I noticed that in docker, you only specify a subnet and mask. This makes sense because if you are deploying this on AWS, a VPC will be defined already, all you need to do is create a new subnet mask for your containerized environment.

# Terminal-1 Ubuntu
docker network create \
-o com.docker.network.bridge.name=docker1 \
--subnet=172.18.0.0/24 \
--gateway=172.18.0.253 \
custom_bridge
Enter fullscreen mode Exit fullscreen mode

Verify Network 

# Terminal-1 Ubuntu

# This is a simulated command and output
ubuntu@ubuntu:~$ docker network ls
NETWORK ID     NAME            DRIVER    SCOPE
6e5ffkvms8c3   bridge          bridge    local
**d3b0029faed7   custom_bridge   bridge    local**
e9f55dsdf605   host            host      local
e9708nj5179a   none            null      local
Enter fullscreen mode Exit fullscreen mode

Pull Alpine Image

I love using Alpine Linux because it’s lightweight.

# Terminal-1 Ubuntu
docker pull alpine
Enter fullscreen mode Exit fullscreen mode

Open a New Terminal

Execute this command to open a new tab. "⌘ + T"

Then enter the Ubuntu shell

# Terminal-2 Mac
multipass shell ubuntu
Enter fullscreen mode Exit fullscreen mode

Listen for ARP Packets in Each Bridge

Now that the Ubuntu shell has been initialized, execute the below command to capture all packets.

docker0 

# Terminal-2 Ubuntu
sudo tcpdump -i docker0 -w capture_docker_0.pcap
Enter fullscreen mode Exit fullscreen mode

Open a third terminal tab "⌘ + T"

docker1

Execute another command to listen for all packets in the docker1 bridge interface.

# Terminal-3 Ubuntu
sudo tcpdump -i docker1 -w capture_docker_1.pcap
Enter fullscreen mode Exit fullscreen mode

Create 2 Containers in the Default Bridge, Also Connect Them to the Custom Bridge 

# Terminal-2 Ubuntu

# Create containers in the default bridge
docker run -itd \
--name=alpine1 \
--ip=172.17.0.2 \
alpine

docker run -itd \
--name=alpine2 \
--ip=172.17.0.4 \
alpine

# Connect new interfaces in the containers to another network.
docker network connect \
--ip=172.18.0.3 \
custom_bridge alpine1

docker network connect \
--ip=172.18.0.5 \
custom_bridge alpine2
Enter fullscreen mode Exit fullscreen mode

Send Pings to the Internet From the First Interface 

# Ping google.com four times in each container from 'bridge'
docker exec -it alpine1 ping -I 172.17.0.2 -c 2 google.com
docker exec -it alpine2 ping -I 172.17.0.4 -c 2 google.com
Enter fullscreen mode Exit fullscreen mode

Send Pings to the Internet From the Second Interface 

# Ping google.com four times in each container from 'bridge'
docker exec -it alpine1 ping -I 172.18.0.3 -c 2 google.com
docker exec -it alpine2 ping -I 172.18.0.5 -c 2 google.com
Enter fullscreen mode Exit fullscreen mode

View MAC Addresses of Each Bridge Interface

The Organizationally Unique Identifier (OUI) of all Docker network adapters is 02:42. So expect all docker container MAC addresses to begin with that.

# Terminal-1 Ubuntu
ip --brief link | grep -E 'docker0|docker1' | awk '{print $1, $3}'
Enter fullscreen mode Exit fullscreen mode

Output

docker0 02:42:28:a8:cb:f5
docker1 02:42:7c:61:6d:f0
Enter fullscreen mode Exit fullscreen mode

View Mac Addresses of Each Container Interface

bridge 

# Terminal-1 Ubuntu
docker network inspect bridge --format '{{range .Containers}}{{.Name}}: {{.MacAddress}}{{"\n"}}{{end}}'
Enter fullscreen mode Exit fullscreen mode

Output-1

alpine1: 02:42:ac:11:00:02
alpine2: 02:42:ac:11:00:03
Enter fullscreen mode Exit fullscreen mode

custom_bridge 

# Terminal-2 Ubuntu
docker network inspect custom_bridge --format '{{range .Containers}}{{.Name}}: {{.MacAddress}}{{"\n"}}{{end}}'
Enter fullscreen mode Exit fullscreen mode

Output-2

alpine1: 02:42:ac:12:00:03
alpine2: 02:42:ac:12:00:05
Enter fullscreen mode Exit fullscreen mode

End ARP Packet Capture 1 

# Terminal-2 Ubuntu
"control + c"
Enter fullscreen mode Exit fullscreen mode

End ARP Packet Capture 2 

# Terminal-3 Ubuntu
"control + c"
Enter fullscreen mode Exit fullscreen mode

Move the Files to the Local Directory

The captured packet files will be moved to the local directory so they can be viewed and analyzed using Wireshark.

# Terminal-1, 2, or 3 Ubuntu
mv capture_docker_0.pcap /mnt
mv capture_docker_1.pcap /mnt
Enter fullscreen mode Exit fullscreen mode

View Packet Captures

Docker0 Bridge Interface

Now I located and opened the packet capture.

docker0

We noticed something interesting: The default MAC addresses of alpine1: 02:42:ac:11:00:02, and alpine2: 02:42:ac:11:00:03, were requesting the MAC address for the bridge's interface, docker0: 02:42:28:a8:cb:f5, to send an Ethernet frame to google.com. The bridge's interface was used as the next hop.

We also see that the DNS name server lookup for google.com could be possible only after an ARP reply from docker0: 02:42:28:a8:cb:f5.

Docker1 Bridge Interface

docker1

The same also applies here. the custom MAC addresses of alpine1: 02:42:ac:12:00:03, and
alpine2: 02:42:ac:12:00:05 were requesting the mac-address for the bridge’s interface docker1: 02:42:7c:61:6d:f0 so it can send an ethernet frame destined to google.com. The bridge’s interface is used as the next hop.

Go to, View Mac Addresses of Each Bridge Interface and View Mac Addresses of Each Container Interface to confirm the mac-addresses of each device when analyzing the packet capture.

Clean-Up

Stop and Remove the Container 

docker stop alpine1 alpine2
docker rm alpine1 alpine2
Enter fullscreen mode Exit fullscreen mode

Delete Custom Bridge 

docker network rm custom_bridge
Enter fullscreen mode Exit fullscreen mode

Verify Network List 

root@ubuntu:/home/ubuntu# docker network ls
NETWORK ID     NAME      DRIVER    SCOPE
2eabde4e866c   bridge    bridge    local
e9f54b21c605   host      host      local
e9708605179a   none      null      local
Enter fullscreen mode Exit fullscreen mode

Stop Ubuntu Server 

multipass stop ubuntu
Enter fullscreen mode Exit fullscreen mode

Conclusion

Now, I understand that everyone can't stop talking about Kubernetes, but a lot of senior engineers have advised that it'd be best to learn docker before picking kubernetes up. Though I've played with Kubernetes severally, I struggled. However, each new day I keep spending with docker makes understanding kubernetes a piece of cake. This guide serves as a strong foundation for analyzing ARP packets in containers.

Resources

I enjoyed these two articles from Hank Preston, a principal engineer at Cisco. The last one is from me.

Top comments (0)

Read next

kalkwst profile image

A Deep Dive into Container Identification and Dependency Management

Kostas Kalafatis -

jao_dev profile image

Docker for beginners: Creating Database Containers

João Vitor -

ietxaniz profile image

How to implement HTTPS in Local Networks Using Let's Encrypt

Iñigo Etxaniz -

ken_mwaura1 profile image

Troubleshooting Common Issues in Kubernetes Deployments

Zoo Codes -