DEV Community

Cover image for How to Secure a Docker Host Using Firewalld
Sören Metje
Sören Metje

Posted on • Edited on

How to Secure a Docker Host Using Firewalld

#docker #firewall #devops #linux

If you are using a firewall like ufw or firewalld and docker you may encounter the problem that docker bypasses the firewall rules.

Goal

  • The firewall rules should count for whole host system - so including Docker containers with port mappings
  • A Docker container should be accessible from the internet if and only if the host port used in Docker container port mapping is allowed in the firewall
  • The approach should not break container networking

Existing Approaches

I found following approaches that try to fix the problem. However, each approach introduced another problem:

  • Just do not use docker. Podman for example obeys firewall rules by default. Problem: Some can not or may not want to switch to a different container runtime, but I generally recommend checking if this is an option for you. This article includes a comparison.
  • Use external firewall like security groups in Openstack instead of ufw or firewalld. Problem: Not available in my case.
  • Just do not map ports in docker. Problem: may introduce security risk because of no single source of truth for exposed host ports.
  • Disabling iptables for docker. Problem: Containers can not access internet.
  • Configuring the firewall to ignore port mappings. Problem: The port inside the container have to be allowed in the host firewall. If multiple containers use same port and only one should be allowed we have to additionally specify container IP or service name. In summary: Counterintuitive, complex and error-prone.

Approach

Idea: Disable iptables for docker and configure firewalld to allow container networking. It is based on this Medium article by Erfan Sahafnejad and several posts. I tested it with Ubuntu 22.04.2 LTS but the concept should also work on other Linux.

Security Implications

It is important to notice, that this approach can have security implications depending on the setup, e.g., as described Keval Kapdee's post. In his setup, he operated a mail server in a docker container with a similar configuration as discussed in this article. Due to the configured masquarading of packets, from the mailserver perspective, all packets originate from the IP address 172.22.1.1, which is listed as a trusted address in Postfix by default. Therefore, Postfix relayed all requests from every internet IP address as these were seen as originating from a trusted address. In summary, this approach is not suited for use cases, which use original internet IP addresses, e.g., for access control.

Overall, especially in production setups, I recommend using other approaches such as Podman instead of the approach discussed in this article.

Preparation

If you added any configuration to iptables regarding docker before, remove it first.

If ufw is installed and active, disable it:

ufw disable
Enter fullscreen mode Exit fullscreen mode

Install and activate firewalld:

apt update && apt install firewalld -y
systemctl enable --now firewalld

# Confirm that the service is running
firewall-cmd --state
Enter fullscreen mode Exit fullscreen mode

Compared to ufw, firewalld is more powerful - it provides features that we need for upcoming firewall configurations. However, it also just a convenient frontend for iptables. Learn more about firewalld here: https://docs.rockylinux.org/guides/security/firewalld-beginners/

Disable iptables for Docker

Disable iptables for docker in /etc/docker/daemon.json so it should look like follows:

{
"iptables": false
}
Enter fullscreen mode Exit fullscreen mode

If /etc/docker/daemon.json does not exist, create the file first.

Restart docker:

systemctl restart docker
Enter fullscreen mode Exit fullscreen mode

Already at this point, only container ports that are allowed in firewall should be reachable from the internet. However, as a side effect of disabling iptables in docker, we broke container internet access: From the inside of containers we can not access the internet anymore.

docker run --rm busybox ping -c4 8.8.8.8
# PING 8.8.8.8 (8.8.8.8): 56 data bytes
#
# --- 8.8.8.8 ping statistics ---
# 4 packets transmitted, 0 packets received, 100% packet loss
Enter fullscreen mode Exit fullscreen mode

Configure firewalld

At next, we configure firewalld to enable docker container networking.

Add Masquerading to the zone which leads out to the Internet, typically public:

# Masquerading allows for docker ingress and egress (this is the juicy bit)
firewall-cmd --zone=public --add-masquerade --permanent
# Reload firewall to apply permanent rules
firewall-cmd --reload
Enter fullscreen mode Exit fullscreen mode

Sources: https://serverfault.com/a/987687 and https://serverfault.com/a/1046550

Additionally, in order to enable docker containers accessing host ports, add docker interface to the trusted zone:

# Show interfaces to find out docker interface name
ip link show

# Assumes docker interface is docker0
firewall-cmd --permanent --zone=trusted --add-interface=docker0
firewall-cmd --reload
systemctl restart docker
Enter fullscreen mode Exit fullscreen mode

Sources: https://unix.stackexchange.com/a/225845 and https://unix.stackexchange.com/a/333356

So far, docker containers that are not attached to a docker network can access the internet. But containers that are attached can still not. This is often the case when using docker compose.

If we try to ping Google DNS server this is the result:

docker run --rm busybox ping -c4 8.8.8.8
# PING 8.8.8.8 (8.8.8.8): 56 data bytes
# 64 bytes from 8.8.8.8: seq=0 ttl=58 time=3.699 ms
# 64 bytes from 8.8.8.8: seq=1 ttl=58 time=3.588 ms
# 64 bytes from 8.8.8.8: seq=2 ttl=58 time=3.587 ms
# 64 bytes from 8.8.8.8: seq=3 ttl=58 time=3.518 ms
#
# --- 8.8.8.8 ping statistics ---
# 4 packets transmitted, 4 packets received, 0% packet loss
# round-trip min/avg/max = 3.518/3.598/3.699 ms


# Create docker network for testing purpose (can be deleted later)
docker network create --driver bridge mynet

docker run --rm --net mynet busybox ping -c4 8.8.8.8
# PING 8.8.8.8 (8.8.8.8): 56 data bytes
#
# --- 8.8.8.8 ping statistics ---
# 4 packets transmitted, 0 packets received, 100% packet loss
Enter fullscreen mode Exit fullscreen mode

To fix this, add your network interface to public zone:

# Show public ip
curl -4 ip.gwdg.de
# Show interfaces to find out network interface name with your public IP
ip addr

# Assumes network interface with your public IP is eth0
# (ens18 is also a name I came accross)
firewall-cmd --permanent --zone=public --add-interface=eth0
firewall-cmd --reload
Enter fullscreen mode Exit fullscreen mode

Networking should work properly now and therefore containers should be able to access the internet.

If we now try to ping Google DNS server again, it works as expected:

docker run --rm busybox ping -c4 8.8.8.8
# PING 8.8.8.8 (8.8.8.8): 56 data bytes
# 64 bytes from 8.8.8.8: seq=0 ttl=58 time=3.641 ms
# 64 bytes from 8.8.8.8: seq=1 ttl=58 time=3.565 ms
# 64 bytes from 8.8.8.8: seq=2 ttl=58 time=3.605 ms
# 64 bytes from 8.8.8.8: seq=3 ttl=58 time=3.546 ms
#
# --- 8.8.8.8 ping statistics ---
# 4 packets transmitted, 4 packets received, 0% packet loss
# round-trip min/avg/max = 3.546/3.589/3.641 ms

docker run --rm --net mynet busybox ping -c4 8.8.8.8
# PING 8.8.8.8 (8.8.8.8): 56 data bytes
# 64 bytes from 8.8.8.8: seq=0 ttl=58 time=3.671 ms
# 64 bytes from 8.8.8.8: seq=1 ttl=58 time=3.644 ms
# 64 bytes from 8.8.8.8: seq=2 ttl=58 time=3.561 ms
# 64 bytes from 8.8.8.8: seq=3 ttl=58 time=3.508 ms
#
# --- 8.8.8.8 ping statistics ---
# 4 packets transmitted, 4 packets received, 0% packet loss
# round-trip min/avg/max = 3.508/3.596/3.671 ms
Enter fullscreen mode Exit fullscreen mode

Open Firewall Ports

In the end, open the desired ports for your service to allow incoming traffic, e.g. on port 8080:

firewall-cmd --permanent --zone=public --add-port=8080/tcp
# Reload firewall to apply permanent rules
firewall-cmd --reload
Enter fullscreen mode Exit fullscreen mode

Extra: Testing

It is important to run tests to ensure the whole setup is working properly. Although the actual tests depend on your setup, here are some statements that may be important to verify:

  • Container running on allowed port can be accessed from internet
  • Container running on not allowed port can not be accessed from internet
  • Container can access internet
  • Container with new docker network can access internet
  • Container can access service running on host system port
  • Container can access other container inside same docker network

To start a webserver on 8080 you can run:

docker run --restart unless-stopped -p 8080:80 -d nginx
Enter fullscreen mode Exit fullscreen mode

To curl a service running on host system port from inside of a container you can run:

# Show docker interface ip address
ip addr
# ...
# 3: docker0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default 
#    link/ether 02:42:37:6b:6e:2a brd ff:ff:ff:ff:ff:ff
#    inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
#       valid_lft forever preferred_lft forever
# ...

# Assumes docker interface ip is 172.17.0.1
# and service runs on port 80
docker run --rm curlimages/curl -v http://172.17.0.1:80/
Enter fullscreen mode Exit fullscreen mode

Further Reading

Sources

Top comments (8)

Collapse
 
wget profile image
William Gathoye •

Thanks a lot! This is the solution I have been searching for months. The only way to circumvent the problem so far was to define a security rule on the firewall at the cloud provider level.

Your solution is what is fixing my problem completely. Thank you.

Collapse
 
raketenberd profile image
raketenberd •

Thank your for the great post!

Does setting masquerade on the public interface bring any security implications?
Traffic reaching the 'eth0' interface with a destination other the server ip forwarded now instead of filtered. When this traffic originates from the docker interface/ips forwardind is great but how is it the other way around?
Does traffic from the internet with destination e.g. one of your docker containers is routed as well bypassing the firewall filter rules?

Collapse
 
soerenmetje profile image
Sören Metje •

So far, I am not aware of any specific vulnerabilities introduced by masquerading. However, this may depend on the actual usage / setup. I strongly recommend to extensively check whether the setup works as intended.

Regarding traffic from the internet with destination container: Packets from the internet with destination set to a container IP address, do not reach your server in first place. Container IP addresses are only accessible within the local network on the server. This article explains this among other interesting details.
If the server IP address is set as destination in a packet, the firewall rules do apply.

Collapse
 
thechubbypanda profile image
Keval Kapdee •

See my reddit post for a reason why this does have security implications :_)

Collapse
 
soerenmetje profile image
Sören Metje •

You are right. Thank you for sharing! I will update the article to include the implications described in your post.

Collapse
 
sblantipodi profile image
Davide Perini •

@soerenmetje it seems to work very well but in this way I can't use fail2ban...
containers isn't able to log the external ip addresses...
is there a workaround for fail2ban?

Collapse
 
soerenmetje profile image
Sören Metje •

I haven't dealt with that yet. If you find a workaround, feel free to post it here.

Read next

dgwhited profile image

Terraforming Resource Control Policies

Dustin Whited -

quanphat profile image

How to integrate Elasticsearch in Express

Le Quan Phat -

signadot profile image

Why Shift Testing Left Part 2: QA Does More After Devs Run Tests

Signadot -

idsulik profile image

Introducing Helm CEL: A More Expressive Way to Validate Your Helm Charts

Suleiman Dibirov -