Raspberry Pi 4 Devbox

Playing around with Docker containers I wanted to set up a Raspberry Pi 4 as the ultimate host for various projects. In this post I'll try to explain how I turned my Raspberry Pi exactly into that ultimate host.

The first problem is that I want to have multiple services exposed on their respective standard ports, but if I assign ports the usual way like -p 12345:12345 I can expose only one service on that port. And that is not good enough.

Multiple static IPs

My Pi is set to always receive 192.168.1.40 from dhcp and I know that there is nothing from 41 to 49 so I can use that set. To assign them I created /etc/dhcpcd.exit-hook with the following content

# just eth0 interface
[ "${interface}" = "eth0" ] || exit 0

# after dhcp address is assigned
[ "${reason}" = BOUND ] || exit 0

# only if there are no secondary addresses assigned
! ip addr show | grep -q 'secondary eth0:' || exit 0

ip addr add 192.168.1.41/24 dev eth0 label eth0:1
ip addr add 192.168.1.42/24 dev eth0 label eth0:2
ip addr add 192.168.1.43/24 dev eth0 label eth0:3
ip addr add 192.168.1.44/24 dev eth0 label eth0:4
ip addr add 192.168.1.45/24 dev eth0 label eth0:5
ip addr add 192.168.1.45/24 dev eth0 label eth0:5
ip addr add 192.168.1.46/24 dev eth0 label eth0:6
ip addr add 192.168.1.47/24 dev eth0 label eth0:7
ip addr add 192.168.1.48/24 dev eth0 label eth0:8
ip addr add 192.168.1.49/24 dev eth0 label eth0:9

Ok, so after Pi gets its IP address via DHCP, it will assign itself additional 9 IP addresses. And now I have 10 different addresses at my disposal.

For example, I can run one database on 192.168.1.41

docker run -d --name postgres12 \
--restart unless-stopped \
-v /srv/postgres12/data:/var/lib/postgresql/data \
--env-file /srv/postgres12/vars.env \
--network skynet \
-p 192.168.1.41:5432:5432 \
postgres:12-alpine

and the other one on 192.168.1.46

docker run -d --name postgres11 \
--restart unless-stopped \
-v /srv/postgres11/data:/var/lib/postgresql/data \
--env-file /srv/postgres11/vars.env \
--network groundnet \
-p 192.168.1.46:5432:5432 \
postgres:11-alpine

And both of them are exposed on my local network for easier management (using pgAdmin or something). This example shows two different pgsql versions, 11 and 12, but it can be one IP for a database, another for a web server.

Web servers

Now, talking about web servers I can easily run

docker run -d --name mycoolserver \
--restart unless-stopped \
-v /srv/mycoolserver/server:/opt/mycoolserver \
-v /srv/mycoolserver/data:/var/opt/mycoolserver \
-w="/opt/mycoolserver" \
--network skynet -p 192.168.1.42:80:80 mcr.microsoft.com/dotnet/core/aspnet dotnet MyCoolServer.dll

and now I have aspnet core site exposed as well. Given that this is default webapi project, if I open http://192.168.1.42/weatherforecast in a browser, I should get some random forecast data.

Or just to make it even simpler, using PHP development server

Create file/srv/phptest/index.php with just

<?php phpinfo(); ?>

and run it with

docker run -d --name phptest \
-v /srv/phptest:/srv/www \
--network skynet \
php:alpine php -S 0.0.0.0:80 -t /srv/www

This one is not exposed on the host, so it is time to make it available somehow.

Reverse proxy - Traefik

Using Traefik as a reverse proxy, all those internal services can be exposed through one IP address.

Traefik provides excellent discovery using docker api, but since this is just a setup for various projects, I prefer to set everything manually. Configuration file, traefik.toml is defining only the file provider that listens for file changes in the directory. This allows me to put additional configuration file in that dir or change something and Traefik will almost instantly refresh the configuration.

[entryPoints]
  [entryPoints.web]
    address = ":80"
  [entryPoints.websecure]
    address = ":443"

[providers]
  [providers.file]
    directory = "/etc/traefik/conf"
    watch = true

[api]
  insecure = true

[log]
  level = "DEBUG"

[accessLog]

Starting it with docker

docker run -d --name traefik \
--restart unless-stopped \
-v /srv/traefik/traefik.toml:/etc/traefik/traefik.toml \
-v /srv/traefik/conf:/etc/traefik/conf \
--network skynet \
-p 192.168.1.43:8080:8080 \
-p 192.168.1.43:443:443 \
-p 192.168.1.43:80:80 \
traefik

With Traefik in place, I want to access it using domain name rather than IP address. To do that, I can put a line

192.168.1.43 devbox

into hosts file (c:\Windows\System32\drivers\etc\hosts on Windows or /etc /hosts on linux), but since my router is running dnsmasq I'll put

address=/devbox/192.168.1.43

into dnsmasq custom configuration. This will also work for subdomains as opposed to hosts file where every domain must be set independently.

And now I can access Traefik's dashboard using https://devbox:8080. Great, moving on.

Dynamic configuration requires defining routers, entrypoints, services etc, but the great thing is that it also supports templating. Having a template, I can just duplicate the file and change two variables, name and domain.

{{ $name := "<container_name>"}}
{{ $domain := "<domain_name>" }}
[http.routers]
  [http.routers.{{$name}}-http]
    entryPoints = ["web"]
    rule = "Host(\"{{$name}}.{{$domain}}\")"
    middlewares = ["https-only"]
    service = "{{$name}}"
  [http.routers.{{$name}}-https]
    entryPoints = ["websecure"]
    rule = "Host(\"{{$name}}.{{$domain}}\")"
    service = "{{$name}}"
    [http.routers.{{$name}}-https.tls]

[[http.services.{{$name}}.loadBalancer.servers]]
  url = "http://{{$name}}/"

Using this template I can create phptest.toml. Note that https-only middleware is removed to allow standard http connection. Https will be explained later on.

{{ $name := "phptest"}}
{{ $domain := "devbox" }}
[http.routers]
  [http.routers.{{$name}}-http]
    entryPoints = ["web"]
    rule = "Host(\"{{$name}}.{{$domain}}\")"
    service = "{{$name}}"
  [http.routers.{{$name}}-https]
    entryPoints = ["websecure"]
    rule = "Host(\"{{$name}}.{{$domain}}\")"
    service = "{{$name}}"
    [http.routers.{{$name}}-https.tls]

[[http.services.{{$name}}.loadBalancer.servers]]
  url = "http://{{$name}}/"

Right. So this creates two routers, one for http, one for https, sets appropriate entry point and creates a service behind load balancer which must exist in Traefik even for one server. And Traefik will understand template as:

[http.routers]
  [http.routers.phptest-http]
    entryPoints = ["web"]
    rule = "Host(\"phptest.devbox\")"
    service = "phptest"
  [http.routers.phptest-https]
    entryPoints = ["websecure"]
    rule = "Host(\"phptest.devbox\")"
    service = "phptest"
    [http.routers.phptest-https.tls]

[[http.services.phptest.loadBalancer.servers]]
  url = "http://phptest/"

Using the same name for service and subdomain makes things easy. But if needed it can also be changed easily.

Trying to access http://phptest.devbox in a browser presents standard PHP information page.

HTTPS

All good from the http perspective, but nowadays everything is https. And trying to access https://phptest.devbox will make a browser scream "not secure". The first thought would be to install Teaefik's default certificate, or maybe even create your own certificate (self-signed, local ca, ...). But the problem with that is that if you want to test a website from a mobile phone, it will again scream about invalid certificate, and installing additional certificates to the phones and other devices is just painful.

Alternative to that is to use Let's Encrypt certificate which is usually trusted. But to use Let's Encrypt we need a valid domain. Time to give up on the devbox domain and buy a regular tld. With plenty of gTLDs to choose from and given that this will be used mostly internally, the suggestion is to pick something short and cheap.

Once domain is bought (let's call it devbox.tld instead of standard example.com), depending on the domain registrar, Traefik can acquire certificate on it's own. But again, this is a playground for developing/testing multiple projects, so it would make sense to let another service take care of certificates in case some services don't go through Traefik.

Acme.sh

Acme.sh is a pure Unix shell script implementing ACME client protocol to acquire Let's Encrypt certificates. https://github.com/Neilpang/acme.sh

There isn't a docker container for Raspberry Pi ready on docker hub, but it can be easily built from source. In an empty folder run:

git clone https://github.com/Neilpang/acme.sh.git .
docker build -t neilpang/acme.sh:arm32v7 .

After that start it in a docker container

docker run -itd --name acme.sh \
--restart unless-stopped \
-v "/srv/certs":/acme.sh \
neilpang/acme.sh:arm32v7 daemon

Request a certificate for both the main domain plus the wildcard for that domain. This will cover devbox.tld, phptest.devbox.tld, mycoolserver.devbox.tld etc. Having a single wildcard certificate is just nice and clean.

docker exec acme.sh --issue --dns <your_registrar> -d 'devbox.tld' -d '*.devbox.tld'

Putting it all together

Now that the certificate is acquired, it is time to put everything together.

In all docker run commands I mounted local folders instead of letting docker create volumes. This is because I prefer to have an easy access to all configurations, data, etc. while playing around. This probably isn't something that should go into production, but for developing and testing various scenarios, it makes it easy to manage. Certificates are in /srv/certs directory, traefik is in /srv/traefik, mycoolserver is in /srv/mycoolserver etc.

Traefik

Back to the Traefik. Since the original docker run command line does not include certificates folder, container can be stopped, deleted and run again mounting certificates as

docker run -d --name traefik \
--restart unless-stopped \
-v /srv/traefik/traefik.toml:/etc/traefik/traefik.toml \
-v /srv/traefik/conf:/etc/traefik/conf \
-v /srv/certs:/etc/traefik/ssl \
--network skynet \
-p 192.168.1.43:8080:8080 \
-p 192.168.1.43:443:443 \
-p 192.168.1.43:80:80 \
traefik

In configuration directory, create new default_ssl.toml to load certificates and to set http to https redirection middleware.

[http.middlewares]
  [http.middlewares.https-only.redirectScheme]
    scheme = "https"

[[tls.certificates]]
  certFile = "/etc/traefik/ssl/devbox.tld/fullchain.cer"
  keyFile  = "/etc/traefik/ssl/devbox.tld/devbox.tld.key"

Finally phptest.toml, now with proper TLD and active middleware.

{{ $name := "phptest"}}
{{ $domain := "devbox.tld" }}
[http.routers]
  [http.routers.{{$name}}-http]
    entryPoints = ["web"]
    rule = "Host(\"{{$name}}.{{$domain}}\")"
    middlewares = ["https-only"]
    service = "{{$name}}"
  [http.routers.{{$name}}-https]
    entryPoints = ["websecure"]
    rule = "Host(\"{{$name}}.{{$domain}}\")"
    service = "{{$name}}"
    [http.routers.{{$name}}-https.tls]

[[http.services.{{$name}}.loadBalancer.servers]]
  url = "http://{{$name}}/"

Router

On router, under dnsmasq configuration add

address=/devbox.tld/192.168.1.43

(Or in hosts add)

192.168.1.43 devbox.tld
192.168.1.43 phptest.devbox.tld

And that's it. Opening http://devbox.tld:8080 presents Traefik's dashboard, http://phptest.devbox.tld redirects to https://phptest.devbox.tld and presents PHP info page. And everything works on a mobile phone without warnings or errors.

Final thoughts

The idea behind this was to set up a playground for developing and testing various scenarios, web applications, services, etc. in a simple inexpensive environment.

Instead of Raspberry Pi, this can be run in a Hyper-V linux machine. Minimal debian installation + docker on a Hyper-V guest that is connected to external virtual switch should do the trick.

Instead of using domain just internally, setting up dynamic dns + cname for the bought domain plus forwarding port 443 on a router to traefik will allow external access for testing webhooks or similar.

Every container is started on its own instead of using docker compose because I can easily stop one, leaving others unaffected. I can swap Traefik with Nginx or Acme.sh with Certbot, or something else. And by mounting volumes, data is persisted and easily accessible on Pi (or even on NAS if it is mounted on Pi).

Networks are also defined as needed, so a couple of containers can be on one network, a couple on another, etc.

And setting up multiple static ip addresses on a docker host allows it to be the "whole corporate network" in a very small package.

Anyway, I hope you find this idea interesting.