Table of Contents
- Introduction
- Requirements
-
Architecture Deployment Guide
- Enter into the Shell of the Ubuntu Server Again
- Create a New Network
- Verify Network
- Pull Alpine Image
- Listen for ARP Packets in each Bridge
- Create 2 Containers in the Default Bridge, Also Connect Them to the Custom Bridge
- Send Pings to the Internet From the First Interface
- Send Pings to the Internet From the Second Interface
- View Mac Addresses of Each Bridge Interface
- View Mac Addresses of Each Container Interface
- End ARP Packet Capture 1
- End ARP Packet Capture 2
- Move the Files to the Local Directory
- View Packet Captures
- Clean-Up
- Conclusion
- Resources
Read this as technical documentation instead.
Introduction
I've been immersing myself in CI/CD pipeline studies lately, and now that I've gotten a good grasp of it, I can finally dedicate my full attention to understanding container technologies. To start, I decided to focus on networking, which is the stack I'm most familiar with.
In this simple how-to guide, I'll walk you through observing the Address Resolution Protocol (ARP) in action within a Docker environment.
An Address Resolution Protocol(ARP) allows a container to learn the MAC address of another device (in this lab, it’s the bridge and other containers) dynamically.
Without an ARP, a ping between containers, external networks, or even a web request between containers and other devices fails.
Requirements
I recently made the switch to a Mac and found using Multipass from Canonical simpler than spinning up a VM with tools like Virtualbox. However, when it comes to container networking labs, non-Linux systems are not recommended. While I downloaded Docker Desktop for Mac, I had trouble seeing all the Docker network interfaces.
For Windows users, I will advise you to use WSL2 instead, it’s easier to deploy and manage compared to having to use a virtual machine like Virtualbox.
Keep in mind that the Ubuntu installation requirement is only for Mac and Linux users. This guide will use three terminal tabs throughout.
Install Homebrew and Multipass
Download Homebrew
# Terminal-1 Mac/Linux
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
Download Multipass
# Terminal-1 Mac/Linux
brew install multipass
Download an Ubuntu Server from Multipass
This Ubuntu server will be customized to run the same specifications as an AWS EC2-T2 Micro instance type.
- 1 CPU
- 1 GB of RAM
- 8 GB of disk
- version 22.04
Simple and fast, right?
# Terminal-1 Mac/Linux
multipass launch jammy --name=ubuntu --cpus=1 --disk=8G --memory=1G
jammy
is the image name for Ubuntu server version 22.04.
Go to the Shell of the Ubuntu Server
# Terminal-1 Mac/Linux
multipass shell ubuntu
Install Docker
Download docker on the Ubuntu server.
# Terminal-1 Ubuntu
sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
# Close this Ubuntu shell.
# Sometimes you might need to use the exit command severally
# to successfully exit the shell.
exit
Mount a Local Folder to an Ubuntu Directory
Mount the Documents/
folder in your Mac, or other machine to the mnt/
directory in the Ubuntu server running inside Multipass.
# Terminal-1 Mac/Linux
multipass mount /Users/apple/Documents /mnt/
Verify Ubuntu Server Installation
apple@Charles-MBP ~ % multipass info ubuntu
Name: ubuntu
State: Running
Snapshots: 0
IPv4: 192.168.64.4
172.17.0.1
Release: Ubuntu 22.04.3 LTS
Image hash: 9dxa2awl28c8 (Ubuntu 22.04 LTS)
CPU(s): 1
Load: 0.00 0.00 0.00
Disk usage: 2.3GiB out of 7.7GiB
Memory usage: 201.4MiB out of 951.6MiB
Mounts: /Users/apple/Documents => /mnt
UID map: 501:default
GID map: 20:default
Install Wireshark
Choose your preferred machine and download.
Architecture Deployment Guide
I have a simple architecture that deploys two docker containers in two different subnets. In docker, you can attach one container to several subnets. This is achieved by a new interface being created and assigned to that subnet.
Enter Into the Shell of the Ubuntu Server Again
# Terminal-1 Mac/Linux
multipass shell ubuntu
Create a New Network
I noticed that in docker, you only specify a subnet and mask. This makes sense because if you are deploying this on AWS, a VPC will be defined already, all you need to do is create a new subnet mask for your containerized environment.
# Terminal-1 Ubuntu
docker network create \
-o com.docker.network.bridge.name=docker1 \
--subnet=172.18.0.0/24 \
--gateway=172.18.0.253 \
custom_bridge
Verify Network
# Terminal-1 Ubuntu
# This is a simulated command and output
ubuntu@ubuntu:~$ docker network ls
NETWORK ID NAME DRIVER SCOPE
6e5ffkvms8c3 bridge bridge local
**d3b0029faed7 custom_bridge bridge local**
e9f55dsdf605 host host local
e9708nj5179a none null local
Pull Alpine Image
I love using Alpine Linux because it’s lightweight.
# Terminal-1 Ubuntu
docker pull alpine
Open a New Terminal
Execute this command to open a new tab. "⌘ + T"
Then enter the Ubuntu shell
# Terminal-2 Mac
multipass shell ubuntu
Listen for ARP Packets in Each Bridge
Now that the Ubuntu shell has been initialized, execute the below command to capture all packets.
docker0
# Terminal-2 Ubuntu
sudo tcpdump -i docker0 -w capture_docker_0.pcap
Open a third terminal tab "⌘ + T"
docker1
Execute another command to listen for all packets in the docker1
bridge interface.
# Terminal-3 Ubuntu
sudo tcpdump -i docker1 -w capture_docker_1.pcap
Create 2 Containers in the Default Bridge, Also Connect Them to the Custom Bridge
# Terminal-2 Ubuntu
# Create containers in the default bridge
docker run -itd \
--name=alpine1 \
--ip=172.17.0.2 \
alpine
docker run -itd \
--name=alpine2 \
--ip=172.17.0.4 \
alpine
# Connect new interfaces in the containers to another network.
docker network connect \
--ip=172.18.0.3 \
custom_bridge alpine1
docker network connect \
--ip=172.18.0.5 \
custom_bridge alpine2
Send Pings to the Internet From the First Interface
# Ping google.com four times in each container from 'bridge'
docker exec -it alpine1 ping -I 172.17.0.2 -c 2 google.com
docker exec -it alpine2 ping -I 172.17.0.4 -c 2 google.com
Send Pings to the Internet From the Second Interface
# Ping google.com four times in each container from 'bridge'
docker exec -it alpine1 ping -I 172.18.0.3 -c 2 google.com
docker exec -it alpine2 ping -I 172.18.0.5 -c 2 google.com
View MAC Addresses of Each Bridge Interface
The Organizationally Unique Identifier (OUI) of all Docker network adapters is 02:42. So expect all docker container MAC addresses to begin with that.
# Terminal-1 Ubuntu
ip --brief link | grep -E 'docker0|docker1' | awk '{print $1, $3}'
Output
docker0 02:42:28:a8:cb:f5
docker1 02:42:7c:61:6d:f0
View Mac Addresses of Each Container Interface
bridge
# Terminal-1 Ubuntu
docker network inspect bridge --format '{{range .Containers}}{{.Name}}: {{.MacAddress}}{{"\n"}}{{end}}'
Output-1
alpine1: 02:42:ac:11:00:02
alpine2: 02:42:ac:11:00:03
custom_bridge
# Terminal-2 Ubuntu
docker network inspect custom_bridge --format '{{range .Containers}}{{.Name}}: {{.MacAddress}}{{"\n"}}{{end}}'
Output-2
alpine1: 02:42:ac:12:00:03
alpine2: 02:42:ac:12:00:05
End ARP Packet Capture 1
# Terminal-2 Ubuntu
"control + c"
End ARP Packet Capture 2
# Terminal-3 Ubuntu
"control + c"
Move the Files to the Local Directory
The captured packet files will be moved to the local directory so they can be viewed and analyzed using Wireshark.
# Terminal-1, 2, or 3 Ubuntu
mv capture_docker_0.pcap /mnt
mv capture_docker_1.pcap /mnt
View Packet Captures
Docker0 Bridge Interface
Now I located and opened the packet capture.
We noticed something interesting: The default MAC addresses of alpine1: 02:42:ac:11:00:02
, and alpine2: 02:42:ac:11:00:03
, were requesting the MAC address for the bridge's interface, docker0: 02:42:28:a8:cb:f5
, to send an Ethernet frame to google.com
. The bridge's interface was used as the next hop.
We also see that the DNS name server lookup for google.com
could be possible only after an ARP reply from docker0: 02:42:28:a8:cb:f5
.
Docker1 Bridge Interface
The same also applies here. the custom MAC addresses of alpine1: 02:42:ac:12:00:03
, and
alpine2: 02:42:ac:12:00:05
were requesting the mac-address for the bridge’s interface docker1: 02:42:7c:61:6d:f0
so it can send an ethernet frame destined to google.com
. The bridge’s interface is used as the next hop.
Go to, View Mac Addresses of Each Bridge Interface and View Mac Addresses of Each Container Interface to confirm the mac-addresses of each device when analyzing the packet capture.
Clean-Up
Stop and Remove the Container
docker stop alpine1 alpine2
docker rm alpine1 alpine2
Delete Custom Bridge
docker network rm custom_bridge
Verify Network List
root@ubuntu:/home/ubuntu# docker network ls
NETWORK ID NAME DRIVER SCOPE
2eabde4e866c bridge bridge local
e9f54b21c605 host host local
e9708605179a none null local
Stop Ubuntu Server
multipass stop ubuntu
Conclusion
Now, I understand that everyone can't stop talking about Kubernetes, but a lot of senior engineers have advised that it'd be best to learn docker before picking kubernetes up. Though I've played with Kubernetes severally, I struggled. However, each new day I keep spending with docker makes understanding kubernetes a piece of cake. This guide serves as a strong foundation for analyzing ARP packets in containers.
Resources
I enjoyed these two articles from Hank Preston, a principal engineer at Cisco. The last one is from me.
- Exploring Default Docker Networking Part 1
- Exploring Default Docker Networking Part 2
- A Routers Intimacy With MAC Addresses (I wrote this a few years ago. It discusses ARP and ICMP in a Cisco environment.)
Top comments (0)