Last month, we launched a new SaaS service called upload.express. It is a fully customizable uploader for anyone who often shares files. Upload.express is fully open-source and our business model is to offer hosting services.
We quickly faced our first problem: how will we manage all the DNS records and SSL certificates?
The first solution that came to our mind was to develop a small micro-service that would call the API of our domain name hosting provider to create the records, and finally use Let’s Encrypt to generate the SSL certificate.
This is something I implemented 3 years ago in one of my last projects and we came across a huge issue: Let’s Encrypt has per-domain rate limits. You can only request 50 subdomain SSL certificates per week. Our project has a free trial plan so we expect to create more than 50 subdomains per week.
In the past, wildcard SSL certificates used to cost $300 but in 2018 everything changed when let’s encrypt announced they would support them. A wildcard certificate is a certificate you can use on all your subdomains.
In example, if you own the domain “example.com”, you can use the same wildcard certificate “*.example.com” on all your subdomains.
Along with wildcard certificates, most DNS providers support wildcard DNS records. By creating an “A” entry named “*.example.com”, that points to your server’s IP address, any request that come from any of your subdomain would redirect to your server’s IP. By doing this, you remove the necessity to implement your own domain-management microservice.
As you grow, you will need more than a single server to handle all the traffic. Also, configuring the SSL certificates at the load balancer level is easier as you do not have to configure all the different types of applications, you will only have to secure the transmission between the user and the load balancer.
On upload.express, I use docker swarm to manage the deployment. You can also use other cluster management systems with traefik, such as kubernetes. In this short tutorial, I’m going to assume you’re using docker swarm, but it should not be hard to adapt it to kubernetes.
First, I created a network to let traefik communicate with the services deployed on my cluster
docker network create --opt encrypted -d overlay webgateway
Then, I used the following stack to deploy traefik.
version: "3.5" services: traefik: image: traefik:1.7 command: - "--api" - "--loglevel=debug" - "--docker" - "--docker.swarmMode" - "--docker.domain=upload.express" - "--docker.watch" - "--configFile=/etc/traefik/traefik.toml" volumes: - /var/run/docker.sock:/var/run/docker.sock - /opt/traefik/traefik.toml:/etc/traefik/traefik.toml - /opt/traefik/acme.json:/etc/traefik/acme/acme.json environment: - AWS_ACCESS_KEY_ID= - AWS_SECRET_ACCESS_KEY= networks: - webgateway - traefik ports: - target: 80 published: 80 - target: 443 published: 443 - target: 8080 published: 8080 mode: host deploy: mode: global placement: constraints: - node.role == manager update_config: parallelism: 1 delay: 10s restart_policy: condition: on-failure networks: webgateway: driver: overlay external: true traefik: driver: overlay
As you can see, I set two environment variables that correspond to my AWS credentials. This is because we need to use the DNS validation to generate the SSL wildcard certificates. As I’m using route53 to manage my domain, I had to provide my AWS credentials. You can see the list of environment variables you need to set depending on your DNS provider here.
The last step is to create the configuration file for traefik. In my swarm stack, I configured the path to “/opt/traefik/traefik.toml”. All you have to do is creating the file and pasting the following configuration.
defaultEntryPoints = ["https","http"] [entryPoints] [entryPoints.http] address = ":80" [entryPoints.http.redirect] entryPoint = "https" [entryPoints.https] address = ":443" [entryPoints.https.tls] [retry] [accessLog] [docker] endpoint = "unix:///var/run/docker.sock" domain = "example.com" watch = true exposedbydefault = false [acme] email = "email@example.com " storage = "/etc/traefik/acme/acme.json" entryPoint = "https" acmeLogging = true onHostRule = true [acme.dnsChallenge] provider = "route53" delayBeforeCheck = 0 [[acme.domains]] main = "*.example.com"
Do not forget to replace the example domain by yours. Now, start your traefik stack by running the following command:
docker stack deploy -c traefik-stack.yml traefik
To finish, for each of the stacks you will deploy in your swarm cluster, set the following labels:
- traefik.port=4000 - traefik.enable=true - traefik.docker.network=webgateway - traefik.backend=service_name - traefik.frontend.rule=Host:service_name.example.com
and link the webgateway network we created earlier like so:
networks: webgateway: external: true
In example, here's the docker stack I'm using to deploy a upload.express instance:
--- version: '3.7' services: uploadexpress: image: uploadexpress/app networks: - webgateway deploy: placement: constraints: - node.role==worker labels: - traefik.port=4000 - traefik.enable=true - traefik.docker.network=webgateway - traefik.backend=maxencehenneron - traefik.frontend.rule=Host:maxencehenneron.upload.express restart_policy: condition: on-failure mode: replicated replicas: 1 networks: webgateway: external: true
That's it! Traefik will automatically pick the correct wildcard certificate and you will be able to reach your service at "service_name.example.com"