Introduction
This tutorial is the second part of the Traefik series. The first part can be found here.
In the previous tutorial, the basic Traefik concepts were explained and we showed a simple Traefik configuration running in standalone Docker. In this tutorial, we are going to cover some advanced concepts such as TLS, authentication and chain middlewares, the Traefik dashboard, Traefik metrics for Prometheus, and healthchecks.
The codebase for this tutorial can be found here. All docker-compose files that appear in the Traefik tutorials can be found here.
Prerequisites
- Docker
- Docker compose
All docker compose files have been tested with Docker 20.10.12 and docker-compose 1.24.0.
Create the required Docker networks
docker network create traefik_public
docker network create socket_proxy
TL;DR
Advanced concepts
TLS
The full docker-compose
file for this section can be found here.
Traefik can be configured to accept incoming HTTPS connections in order to terminate the SSL connections (meaning that it will send decrypted data to the services). It can be configured to use an ACME provider (like Let's Encrypt) for automatic certificate generation. However, we are not going to cover this as there is already a plethora of very informative material on the subject online. In this tutorial, we are going to create our own CA and Traefik certificates and configure Traefik to use them.
Certificates
We need to create our own TLS certificates in order to have encryption in transit and properly secure the communication from and to Traefik. I have already generated some certificates which can be found here.
However, you can create your own certificates by running the following commands:
mkdir -p certs/{ca,traefik}
# Create CA certificates
openssl genrsa -out certs/ca/rootCA.key 4096
openssl req -x509 -new \
-nodes \
-sha256 \
-days 3650 \
-key certs/ca/rootCA.key \
-subj "/C=GR/L=Athens/O=Karvounis Tutorials, Inc./CN=Karvounis Root CA/OU=CA department" \
-out certs/ca/rootCA.pem
# Create Traefik wildcard certificates
openssl genrsa -out certs/traefik/traefik.key 4096
openssl req -new \
-key certs/traefik/traefik.key \
-subj "/C=GR/L=Athens/O=Karvounis Tutorials, Inc./CN=*.karvounis.tutorial/OU=Dev.to" \
-out certs/traefik/traefik.csr
openssl x509 -req \
-sha256 \
-days 365 \
-CA certs/ca/rootCA.pem \
-CAkey certs/ca/rootCA.key \
-CAcreateserial \
-in certs/traefik/traefik.csr \
-out certs/traefik/traefik.crt
The above commands will create the necessary certificates under the following directory structure:
$ tree certs/
certs/
├── ca
│ ├── rootCA.key
│ ├── rootCA.pem
│ └── rootCA.srl
└── traefik
├── traefik.crt
├── traefik.csr
└── traefik.key
The generated traefik certificate is a wildcard certificate for *.karvounis.tutorial
and will cover all the use cases for the tutorials. From now on, we are going to use these certificates in every docker-compose
file.
Service configuration
traefik:
image: traefik:v2.6
command:
# Entrypoints configuration
- --entrypoints.web.address=:80
## Create a new entrypoint called `websecure` that is going to be used for TLS
- --entrypoints.websecure.address=:443
## Forces redirection of incoming requests from `web` to `websecure` entrypoint
## https://doc.traefik.io/traefik/routing/entrypoints/#redirection
- --entrypoints.web.http.redirections.entryPoint.to=websecure
# Docker provider configuration
- --providers.docker=true
- --providers.docker.exposedbydefault=false
- --providers.docker.endpoint=tcp://socket_proxy:2375
- --providers.docker.network=traefik_public
# File provider configuration
- --providers.file.directory=/traefik/config/my_dynamic_conf
# Logging configuration
- --log.level=info
- --log.format=json
ports:
- 80:80
- 443:443
volumes:
- ./certs/traefik:/traefik/config/certs:ro
- ./config.yml:/traefik/config/my_dynamic_conf/conf.yml:ro
networks:
- traefik_public
- socket_proxy
restart: unless-stopped
depends_on:
- socket_proxy
whoami:
image: traefik/whoami:v1.7.1
labels:
- traefik.enable=true
- traefik.http.routers.whoami_route.entrypoints=websecure
- traefik.http.routers.whoami_route.rule=Host(`whoami.karvounis.tutorial`)
- traefik.http.routers.whoami_route.service=whoami_service
- traefik.http.routers.whoami_route.tls=true
- traefik.http.services.whoami_service.loadbalancer.server.port=80
networks:
- traefik_public
Entrypoints configuration
--entrypoints.websecure.address=:443
Defines an entrypoint called websecure
that will listen on port 443 of the Traefik container. This entrypoint is going to be used for all the TLS connections.
--entrypoints.web.http.redirections.entryPoint.to=websecure
It enables permanent redirecting of all incoming requests from the web
entrypoint to the websecure
entrypoint. That means that even if someone tries to send an HTTP request, that request will be redirected to HTTPS.
File provider configuration
TLS certification configuration is part of the dynamic configuration of Traefik. Unfortunately, we cannot use the Docker provider in order to dynamically configure tls certificates using labels. We have to use the File provider instead.
--providers.file.directory=/traefik/config/my_dynamic_conf
Points to the directory where Traefik can load the dynamic configuration from. In our case, we are going to mount a config.yml
file that contains the paths to our certificates.
Volumes
volumes:
- ./certs/traefik:/traefik/config/certs:ro
- ./config.yml:/traefik/config/my_dynamic_conf/conf.yml:ro
Mounts the local ./certs/traefik
folder and its contents (the Traefik certificates) to the /traefik/config/certs
inside the container. Local ./config.yml
file, that contains the dynamic configuration, will be mounted inside the /traefik/config/my_dynamic_conf/
directory, which is the directory that Traefik looks for its dynamic configuration from.
The path of Traefik's public key and private key in the container are /traefik/config/certs/traefik.crt
and /traefik/config/certs/traefik.key
respectively.
Config
The following config file, which can be found here, is used to define the paths in the container for the certificate and the key.
tls:
certificates:
- certFile: /traefik/config/certs/traefik.crt
keyFile: /traefik/config/certs/traefik.key
whoami service labels
We are going to enable TLS for the whoami_route
. This can be achieved by changing the value of the traefik.http.routers.whoami_route.entrypoints
to websecure
(the HTTPS entrypoint) and setting the traefik.http.routers.whoami_route.tls
label to true.
labels:
- traefik.enable=true
- traefik.http.routers.whoami_route.entrypoints=websecure
- traefik.http.routers.whoami_route.rule=Host(`whoami.karvounis.tutorial`)
- traefik.http.routers.whoami_route.service=whoami_service
- traefik.http.routers.whoami_route.tls=true
- traefik.http.services.whoami_service.loadbalancer.server.port=80
Deployment
Deploy the containers by executing the following command:
docker-compose -f docker-compose.tls.yml up -d
Requests
First, we are going to send a request to http://whoami.karvounis.tutorial
.
$ curl -H "Host: whoami.karvounis.tutorial" \
http://localhost
# OR curl http://whoami.karvounis.tutorial
Moved Permanently
The response is Moved Permanently due to the redirection that we configured with this command: --entrypoints.web.http.redirections.entryPoint.to=websecure
. Every request to the HTTP web
entrypoint will be automatically redirected to the HTTPS websecure
entrypoint!
Let's try to directly hit the HTTPS entrypoint:
$ curl --cacert ./certs/ca/rootCA.pem \
https://whoami.karvounis.tutorial
Hostname: 5f1a8f92cd2b
IP: 127.0.0.1
IP: 172.19.0.2
RemoteAddr: 172.19.0.3:53910
GET / HTTP/1.1
Host: whoami.karvounis.tutorial
User-Agent: curl/7.68.0
Accept: */*
Accept-Encoding: gzip
X-Forwarded-For: 172.20.0.1
X-Forwarded-Host: whoami.karvounis.tutorial
X-Forwarded-Port: 443
X-Forwarded-Proto: https
X-Forwarded-Server: 40e915108da5
X-Real-Ip: 172.20.0.1
Success! Sending a request to the HTTPS entrypoint and specifying the public key of the CA, returned the expected response.
Ping
The full docker-compose
file for this section can be found here.
Traefik provides a ping
endpoint that, when enabled, can be used to check the health of the Traefik instance.
Configuration changes
--ping=true
Enables the /ping
healthcheck URL. However, we are not going to expose it using a router. Instead, we are going to use the URL to check the health of the Docker container by leveraging docker-compose's healthcheck option.
healthcheck configuration
Every 10 seconds, Docker is going to execute the command traefik healthcheck --ping
to establish the health of each Traefik instance (docs). If the command is unsuccessful for 3 consecutive times, Docker will mark the container as unhealthy and will restart it.
healthcheck:
# Run traefik healthcheck command
# https://doc.traefik.io/traefik/operations/cli/#healthcheck
test: ["CMD", "traefik", "healthcheck", "--ping"]
interval: 10s
timeout: 5s
retries: 3
start_period: 5s
Deployment
Deploy the containers by executing the following command:
docker-compose -f docker-compose.ping.yml up -d
Check the status of the traefik service by executing:
docker-compose -f docker-compose.ping.yml ps traefik
After a few seconds, traefik service's status will change from starting
to healthy
.
Dashboard
The full docker-compose
file for this section can be found here.
Traefik offers a dashboard where you can view all the active routers, services and middlewares. In this section, we are going to find out how to enable the dashboard and how to configure the routers to be able to access it.
Traefik service configuration
This is the first time we are going to add Docker labels to the traefik
service. They are going to define a new router, called dashboard
, which will only be accessible through TLS.
In order to enable the dashboard
and the api
, you have to add the --api.dashboard=true
to the command
configuration option of the traefik
service.
Service labels
The Docker labels below define a new router called dashboard
. This router uses a Host Based rule as well as two PathPrefix rules to be able to match all the necessary requests. This router is only accessible through the websecure
entrypoint.
labels:
- traefik.enable=true
- traefik.http.routers.dashboard.rule=Host(`traefik.karvounis.tutorial`) && (PathPrefix(`/api`) || PathPrefix(`/dashboard`))
- traefik.http.routers.dashboard.tls=true
- traefik.http.routers.dashboard.entrypoints=websecure
- traefik.http.routers.dashboard.service=api@internal
Deployment
Deploy the containers by executing the following command:
docker-compose -f docker-compose.dashboard.yml up -d
UI
You can access the dashboard by visiting https://traefik.karvounis.tutorial/dashboard/
and the api
at https://traefik.karvounis.tutorial/api/rawdata
.
Tip: Do not forget the trailing slash /
in /dashboard/
!
Authentication
The full docker-compose
file for this section can be found here.
After exposing the dashboard in the previous section, it is clear that we need to secure it and restrict access only to authenticated users.
Traefik offers the following HTTP Authentication middlewares:
In this section, we are going to use the BasicAuth
middleware to secure the dashboard
router and the DigestAuth
to secure the whoami
router.
Configuration changes
traefik
service:
labels:
- traefik.enable=true
- traefik.http.routers.dashboard.rule=Host(`traefik.karvounis.tutorial`) && (PathPrefix(`/api`) || PathPrefix(`/dashboard`))
- traefik.http.routers.dashboard.tls=true
- traefik.http.routers.dashboard.entrypoints=websecure
- traefik.http.routers.dashboard.service=api@internal
# Middlewares
- traefik.http.routers.dashboard.middlewares=dashboard_auth
## Creates 2 authentication middlewares
### `dashboard_auth` is a BasicAuth middleware and is going to be used by the `dashboard` router.
### dashboard:tutorial
- traefik.http.middlewares.dashboard_auth.basicauth.users=dashboard:$$2y$$05$$T/WVjQVqBc24NLUNI/xuVu0V2B.RPY50k2.CCH5JHGInb3EUeaDcO
### `auth` is a DigestAuth middleware and is going to be used by the `whoami_route` router.
### whoami:tutorial
- traefik.http.middlewares.digest_auth.digestauth.users=whoami:traefik:f4ba293a96d5dcf51eb2f03b5931dd96
whoami
service:
labels:
- traefik.enable=true
- traefik.http.routers.whoami_route.entrypoints=websecure
- traefik.http.routers.whoami_route.rule=Host(`whoami.karvounis.tutorial`)
- traefik.http.routers.whoami_route.service=whoami_service
- traefik.http.routers.whoami_route.tls=true
# `whoami_route` uses the `digest_auth` middleware defined in the `traefik` service
- traefik.http.routers.whoami_route.middlewares=digest_auth
- traefik.http.services.whoami_service.loadbalancer.server.port=80
traefik.http.middlewares.dashboard_auth.basicauth.users
This label creates a new BasicAuth
middleware called dashboard_auth
. It contains the user with the credentials dashboard:tutorial
and can contain an array of authorized users. If you have a great number of users, you can also add their credentials to a file, mount that file to the container and specify the usersFile
option to point to that file.
You can generate the passwords with the following ways:
# Using the httpd:2.4-alpine image which is 58.2MB
$ docker run --rm httpd:2.4-alpine htpasswd -nbB dashboard tutorial | sed -e s/\\$/\\$\\$/g
# OR with `xmartlabs/htpasswd` docker image which is 9MB
$ docker run --rm -ti xmartlabs/htpasswd dashboard tutorial | sed -e s/\\$/\\$\\$/g
# OR without docker
$ htpasswd -nbB dashboard tutorial | sed -e s/\\$/\\$\\$/g
dashboard:$$2y$$05$$T/WVjQVqBc24NLUNI/xuVu0V2B.RPY50k2.CCH5JHGInb3EUeaDcO
Tip: when used in docker-compose.yml
, all dollar signs in the hash need to be doubled for escaping!
traefik.http.middlewares.digest_auth.digestauth.users
This label creates a new DigestAuth
middleware called digest_auth
. It contains the user with the credentials whoami:tutorial
and can contain an array of authorized users. The usersFile
option is available here as well.
You can generate the digest credentials with the following commands:
$ print whoami:traefik:$(printf whoami:traefik:tutorial | md5sum | awk '{print $1}')
# OR with htdigest `htdigest [-c] passwordfile realm username` and type the password
$ htdigest -c /tmp/pwd_file traefik whoami && cat /tmp/pwd_file
whoami:traefik:f4ba293a96d5dcf51eb2f03b5931dd96
traefik.http.routers.dashboard.middlewares=dashboard_auth
Instructs the dashboard
router to use the dashboard_auth
as authentication middleware.
traefik.http.routers.whoami_route.middlewares=digest_auth
Instructs the whoami_route
router to use the digest_auth
as authentication middleware.
Tip: You can use middlewares defined in other services!
Deployment
Deploy the containers by executing the following command:
docker-compose -f docker-compose.auth.yml up -d
Requests
If you try to access the https://traefik.karvounis.tutorial/api/rawdata
URL like before, you are going to get a 401
response status code.
$ curl --cacert ./certs/ca/rootCA.pem \
https://traefik.karvounis.tutorial/api/rawdata
401 Unauthorized
In order to access the Traefik api
using curl
, you have to specify the basic auth user credentials.
$ curl --cacert ./certs/ca/rootCA.pem \
-u dashboard:tutorial \
https://traefik.karvounis.tutorial/api/version
{"Version":"2.6.0","Codename":"rocamadour","startDate":"2022-02-23T17:42:57.897252485Z","pilotEnabled":true}
If you want to access the whoami
service, you need to specify the digest credentials of the whoami
authorized user. Otherwise, Traefik is going to respond with a 401
as above.
$ curl --cacert ./certs/ca/rootCA.pem \
--digest -u whoami:tutorial \
https://whoami.karvounis.tutorial
Hostname: 637aadaf37b3
IP: 127.0.0.1
IP: 172.19.0.2
RemoteAddr: 172.19.0.3:40984
GET / HTTP/1.1
Host: whoami.karvounis.tutorial
User-Agent: curl/7.68.0
Accept: */*
Accept-Encoding: gzip
Authorization: Digest username="whoami", realm="traefik", nonce="2ZZJLL5tPk8bEDU8", uri="/", cnonce="YzUxNzMyZWIyODNjN2VlNDIyMDkyMmY3Nzc3YjVkNDE=", nc=00000001, qop=auth, response="518ade9a297beb2f5174e2368e8cf561", opaque="pfm5w5vBpLYhEv7A", algorithm="MD5"
X-Forwarded-For: 172.20.0.1
X-Forwarded-Host: whoami.karvounis.tutorial
X-Forwarded-Port: 443
X-Forwarded-Proto: https
X-Forwarded-Server: 5966f55786bb
X-Real-Ip: 172.20.0.1
Chain middleware
The full docker-compose
file for this section can be found here.
In this section, we are going to create a new Chain middleware and instruct whoami_route
to use it. This middleware consists of two middlewares:
- A new HTTP RateLimit middleware called
simple_ratelimit
. This middleware is going to limit the maximum amount of allowed requests to thewhoami_service
service in a particular time period. -
digest_auth
which we have already seen in the Authentication section.
The chain
middleware is first going to pass the request through the simple_ratelimit
and then through the digest_auth
middleware. If the request manages to successfully pass both, then it will reach the whoami_service
. Traefik's documentation does not specify a hard limit for the amount of middlewares that you can chain.
Configuration changes
We are going to create the new simple_ratelimit
and secured_chain
middlewares by adding the following labels to the traefik
service:
labels:
- traefik.http.middlewares.secured_chain.chain.middlewares=simple_ratelimit,digest_auth
## The `simple_ratelimit` middleware allows
## an average of 5 requests per 5 seconds
## and a burst of 2 requests.
- traefik.http.middlewares.simple_ratelimit.ratelimit.average=5
- traefik.http.middlewares.simple_ratelimit.ratelimit.period=5s
- traefik.http.middlewares.simple_ratelimit.ratelimit.burst=2
Based on the above numbers, the maximum allowed request rate is r=average/period=5/5s=1 request/second
. If we exceed that rate, the requests are going to be automatically rejected with a 429 HTTP status code.
We also need to instruct whoami_route
to use the secured_chain
middleware:
labels:
# Use the `secured_chain` chain middleware
- traefik.http.routers.whoami_route.middlewares=secured_chain
Deployment
Deploy the containers by executing the following command:
docker-compose -f docker-compose.chain.yml up -d
Requests
Sending a simple curl
request to https://whoami.karvounis.tutorial
will work as before. Nothing out of the ordinary yet.
curl --cacert ./certs/ca/rootCA.pem \
--digest -u whoami:tutorial \
https://whoami.karvounis.tutorial
The following command is going to send a curl request to https://whoami.karvounis.tutorial
every 1.5 seconds. All 5 requests are going to succeed because the rate is slower than the maximum allowed request rate.
SLEEP_TIMER=1.5s
for i in {1..5};
do
echo "\nRequest number: $i"
curl --cacert ./certs/ca/rootCA.pem \
--digest -u whoami:tutorial \
https://whoami.karvounis.tutorial
sleep "${SLEEP_TIMER}"
done
On the other hand, the following command is going to send a curl request to https://whoami.karvounis.tutorial
every 0.5 seconds. This request rate of 2 requests/sec is faster that the maximum allowed rate! The very first request will succeed but all the subsequent ones will be rejected with a 429 Too Many Requests
HTTP status code.
SLEEP_TIMER=0.5s
for i in {1..5};
do
echo "\nRequest number: $i"
curl --cacert ./certs/ca/rootCA.pem \
--digest -u whoami:tutorial \
https://whoami.karvounis.tutorial
sleep "${SLEEP_TIMER}"
done
Metrics
The full docker-compose
file for this section can be found here.
Currently, Traefik supports 4 metrics backends:
- Datadog
- InfluxDB
- Prometheus
- StatsD
In this section, we are going to expose Traefik's metrics for Prometheus.
Configuration changes
First, we are going to enable the prometheus backend and disable the default internal router in order to allow one to create a custom router to the prometheus@internal
service.
command:
# Prometheus metrics
## Enable prometheus metrics
- --metrics.prometheus=true
## Create a manual router instead of the default one.
- --metrics.prometheus.manualrouting=true
- --metrics.prometheus.addrouterslabels=true
...
The custom metrics
router exposes the metrics through https://traefik.karvounis.tutorial/metrics
and uses the dashboard_auth
BasicAuth middleware for authentication.
labels:
# metrics
router configuration
- traefik.http.routers.metrics.rule=Host(traefik.karvounis.tutorial
) && PathPrefix(/metrics
)
- traefik.http.routers.metrics.tls=true
- traefik.http.routers.metrics.entrypoints=websecure
- traefik.http.routers.metrics.service=prometheus@internal
- traefik.http.routers.metrics.middlewares=dashboard_auth
...
Deployment
Deploy the containers by executing the following command:
docker-compose -f docker-compose.metrics.yml up -d
Requests
Send the following request to https://traefik.karvounis.tutorial/metrics
. The response contains all the Prometheus metrics for the particular Traefik instance.
curl --cacert ./certs/ca/rootCA.pem </span>
-u dashboard:tutorial </span>
https://traefik.karvounis.tutorial/metrics
Final notes
I hope you enjoyed this advanced tutorial on Traefik. Even more interesting tutorials are on the way! Please, let me know your thoughts in the comments section below! Cheers
Top comments (2)
Good job, thank you.
Hey, I really liked this series. I would love to send you some Traefik Swag: traefik.io/submit-my-contribution/