This article provides a basic overview of how Remote.It's open source tool may be used to discover problems with MTU (maximum transmission unit). Did you know MTU values can by asymmetrical? Do you know how to test for this within your Docker network? Remote.It has written a tool to help identify these issues.
Docker asymmetric MTU
If we run Remote.It or other services in a Docker container running on macOS we see the following:
Connected to ID user@remote.it - f3:25:9f:01:33:b9:f1:ed at 192.168.2.21:37020
Session Max Packet Detected 1450 (MTU) Max Data Tunnel Size 1386 bytes.
Session Max Packet Received 1500 (MRU) Max Data Tunnel Size 1436 bytes.
last packet from Session peer 0 seconds ago
retry at count 0
Note that we have an asymmetric MTU: the container has an MTU of 1450 bytes (allowing for a tunnel payload of 1386 bytes with a header and encryption overhead of 64 bytes) but an MRU of 1500 bytes (with the same 64-byte overhead). In this case, Remote.It has discovered the MTU by sending packets of various lengths. The discovery is a step-by-step process and Remote.It stops when it has found a packet length that works. The discovery algorithm is granular in that Remote.It may stop a few bytes before the actual MTU. If we run a test program and exhaustively search for the exact MTU, we find the Docker MTU on macOS is 1478 bytes (with a UDP data payload of1450 bytes)
Now, If we run Remote.It on the macOS machine we see the following:
Session List
session index 1 of id 0x558C1B10794B03B05571579018B7AB4433F4DD9A is in state 3 encrypt type 3
using spi is 0x33F4DD9A
Connected to ID user@remote.it - 80:00:01:7f:7e:03:c7:3c at 10.60.0.3:63804
Session Max Packet Detected 1500 (MTU) Max Data Tunnel Size 1436 bytes.
Session Max Packet Received 1500 (MRU) Max Data Tunnel Size 1436 bytes.
Session at 19 seconds of 7200 seconds max life
last packet from Session peer 5 seconds ago
retry at count 0
Now we see the full symmetric MTU of 1500 bytes. Note: This connection was made using chacha20 encryption. If we had used a lighter weight encryption, we might see Tunnel Size 1450 bytes.
Looking at the Docker interface on the container running inside macOS, we see the following:
~ # ifconfig
eth0 Link encap:Ethernet HWaddr 02:42:AC:11:00:02
inet addr:172.17.0.2 Bcast:172.17.255.255 Mask:255.255.0.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:34808 errors:0 dropped:0 overruns:0 frame:0
TX packets:40066 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:16585975 (15.8 MiB) TX bytes:9236031 (8.8 MiB)lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:1 errors:0 dropped:0 overruns:0 frame:0
TX packets:1 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:151 (151.0 B) TX bytes:151 (151.0 B)~ #
The container thinks the MTU is 1500 bytes. Without further information, we would think the MTU is symmetric but we have seen above that it is not.
Wireshark
If we look at the packets on the wire using Wireshark for example we can see the MTU.
MTU Bouncer
Remote.It has written and released an open source tool called MTU Bouncer to allow you to check for asymmetric MTU. MTU Bouncer sends a UDP packet back to you with a given length.
See https://github.com/remoteit/MTU_Bouncer
Here is sample output from running MTU Bouncer:
Home-MacBook-Pro:~ user$ nc -u bouncer.remote.it 9999
17
17 (45 MTU)
...Must be between 16 and 1472
^CHome-MacBook-Pro:~ user$
Bouncer uses UDP, so you will have to send response, even a null character, to allow Bouncer know your IP address. So, when you hit return after entering the netcat command, the terminal console will wait for you to enter the response. In the above case, we entered 17 to request Bouncer to send a UDP packet of length 17 with DF=1 (You can also set DF=0, meaning that the bit is not set).
What is MTU?
The OSI model consists of seven abstract layers: Physical, Data link, Network, Transport, Session, Presentation, and Application. The Physical layer may be Ethernet, for example. Ethernet carries frames that carry the packets.
There is a limit on the size of the Ethernet frame that typically limits the size of data to 1500 bytes. This limit of the link layer is called the MTU, maximum transmission unit.
If the IP layer has a datagram to send, and the datagram is larger than the link layer's MTU, IP performs fragmentation, breaking the datagram up into smaller pieces called fragments so that each fragment is smaller than the MTU.
When two hosts on one network communicate with each other, there is a single network MTU. When two hosts communicate using multiple networks, each link can have a different MTU. The important numbers are not the MTUs of the two networks to which the two hosts are connected, but the smallest MTU of any data link that packets traverse between the two hosts. The smallest network MTU is called the path MTU.
RFC 1191 [Mogul and Deering 1990] specifies the path MTU discovery mechanism, a way to determine the path MTU at any time.
The path MTU between any two hosts is not necessarily constant and may depend on the route being used at any time. Routing need not be symmetric so that the route from A to B may not be the route from B to A, and thus the path MTU need not be the same in both directions. In that case the MRU or maximum receive unit may not be the same as the MTU.
A network with an asymmetric MTU is the situation the focus of this article.
What is Docker?
Docker allows you to package and run an application in a loosely isolated secure environment called a container. The isolation and security allows you to run many containers simultaneously on one host. Containers are lightweight and contain everything needed to run the application, so you do not need to rely on what is currently installed on the host. You can easily share containers, and be sure that everyone you share with gets the same container that works in the same way.
A container is a runnable instance of an image. You can create, start, stop, move, or delete a container using the Docker API or CLI. You can connect a container to one or more networks, attach storage to it, or even create a new image based on its current state.
Docker is written in the Go programming language and takes advantage of several features of the Linux kernel to deliver its functionality. Docker uses a technology called namespaces to provide the isolated workspace called the container. When you run a container, Docker creates a set of namespaces for that container.
These namespaces provide the layer of isolation. Each aspect of a container runs in a separate namespace and its access is limited to that namespace.
One of the reasons Docker containers and services are so powerful is that you can connect them together, or connect them to non-Docker workloads. Docker containers and services do not even need to be aware that they are deployed on Docker, or whether their peers are also Docker workloads or not. Whether your Docker hosts run Linux, Windows, or a mix of the two, you can use Docker to manage them in a platform-agnostic way.
Why use Remote.It with Docker?
Remote.It is a powerful connection tool that can connect any two hosts or devices using a peer-peer connection with an agent on each host. Remote.It can also connect using a web-based tool and a proxy to any target host, using a single agent on the target host.
Developers can be used to connect containers within a host, containers in different hosts on the same network, containers on different networks, or even containers in different data centers for example.
Suppose container 1 has a default IP address 172.17.0.1 and container 2 also has a default IP address 172.17.0.1. Remote.It allows you to connect these two containers.
Summary
We have described MTU, how a path can have asymmetric MTU and an example of an asymmetric MTU situation in a Docker container. We described how to discover an asymmetric MTU using Remote.It and the use of MTU Bouncer.
Learn more about how Remote.It makes connections by reading this article.
Top comments (0)