HTTPS for Local Sites

I recently set up pi-hole, a local DNS sinkhole. It redirects lookups for ad-related domains into the void (and any other domain, via a blocklist). This post is not about pi-hole itself though. Rather, it is about adding HTTPs to the local interface.

Let’s define the problem better. The pi-hole runs on a Raspberry Pi, on the local network. It does not listen on the public internet, and its port is not exposed to it. Pi-hole offers an admin interface as a web UI. It is available locally, under its IP address, or a domain that you can configure locally. I would like to have HTTPS, because accessing sensitive data over an unencrypted connection, local as it may be, seems icky.

In order to enable HTTPS, we need some kind of domain, and a certificate for that domain. We also do not want to go for a self-signed certificate solution.

We thus have to solve the following:

• Getting a domain to point to our IP
• Getting a certificate for that domain
• Auto-renewing that certificate when it expires

I will stress the automation part. Historically, I have been really bad at remembering to update certificates, or restart servers after certificates auto-renew. You might have more consistency than I do!

Getting a domain

I decided to use a sub-domain of fotis.xyz, in this case pi-hole.fotis.xyz.

We could then resolve it via a DNS record to our local IP address, or do it per-device via a hosts file. I went with the hosts file, because I only really administrate the pi-hole through my laptop, and I can always access the admin via IP from other devices (albeit unencrypted). This solves the domain issue.

Worrying about DHCP leases and changing IP for the pi-hole is out of scope for this post :)

Getting a certificate for that domain

This is the interesting part, where I had gotten stuck in the past. I want to use LetsEncrypt, because they are awesome LetsEncrypt uses a mechanism called ACME (Automatic Certificate Management Environment) to issue challenges, that the server must answer to prove that you own the domain.

The “regular” challenge mechanism, that I was familiar with at least, is over HTTP. The client uses a well-known address like http://<YOUR_DOMAIN>/.well-known/acme-challenge/<TOKEN> to place the challenge token, which LetsEncrypt then validates. This process can be manual or automated, for example with plugins for popular servers, such as Apache and Nginx.

However, this does not work for sites that are resolved locally. For example, my local Raspberry Pi only listens on local addresses. The HTTP port 80 is not exposed to the internet. An HTTP ACME challenge would fail, because the pi cannot be reached. It also means I must resolve the domain to the local address at a public DNS level, instead of locally.

That’s when I found the DNS Challenge mechanism. The ACME client makes the request, and gets a challenge token to place in a TXT DNS record (e.g. TXT for _acme-challenge). LetsEncrypt then validates the presence of that record, to verify our control. This is promising, because the Pi can communicate outwards, and nothing needs to connect to its HTTP port!

Automation

Now, you could copy this TXT record manually, but that sucks for the short renewal period of LetsEncrypt. Historically, as I mentioned, I have been bad at staying on top of it.

Surprise! Most DNS providers offer an API these days, to add records and so on. So you can perform the addition of the TXT record as part of the challenge pipeline, with some artisanal scripting. This verifies that you have control of the domain, without caring about the ip it resolves to (or that ip needing to be listening on the internet).

To avoid artisanal scripting, and make this reproducible, I used acme.sh, an ACME client for the command line. I have used Certbot in the past, but I was a bit lazy about managing Python this time around! acme.sh has integrations to many DNS providers, including the one I use, DNSimple.

A more applied example

The following is a guide (mostly to myself) about setting this up with pi-hole specifically.

Install acme.sh

First, install acme.sh via their instructions. I used the “pipe to shell” method, but I know not everyone likes that. Remember to restart the terminal session for it to appear in the PATH.

Set up lighttpd

We must respond to the host name that we define (e.g. pi-hole.fotis.xyz) from our Raspberry Pi. pi-hole uses lighttpd for the HTTP UI already, so it made sense to me to reuse it.

I added the following file under /etc/lighttpd/external.conf, adapted from Scott Helme’s post on pi-hole.

# Install this under /etc/lighttpd/external.conf
$HTTP["host"] == "pi-hole.fotis.xyz" {
  # Ensure the Pi-hole Block Page knows that this is not a blocked domain
  setenv.add-environment = ("fqdn" => "true")

  # Enable the SSL engine with a LE cert, only for this specific host
  $SERVER["socket"] == ":443" {
    ssl.engine = "enable"
    ssl.ca-file =  "/home/pi/cert.pem"
    # This file does not come right off acme.sh, it is merged from the two files
    ssl.pemfile = "/home/pi/merged.pem"
    ssl.honor-cipher-order = "enable"
    ssl.cipher-list = "EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH"
    ssl.use-compression = "disable"
    ssl.use-sslv2 = "disable"
    ssl.use-sslv3 = "disable"
  }

  # Redirect HTTP to HTTPS
  $HTTP["scheme"] == "http" {
    $HTTP["host"] =~ ".*" {
      url.redirect = (".*" => "https://%0$0")
    }
  }
}

Getting the certificate

Now we must get the certificate. We want to use the DNS challenge mode of acme.sh.

We first get a token from DNSimple (this will vary on your provider).

I stored it in a .env file locally on the server:

export DNSimple_OAUTH_TOKEN="sdfsdfsdfljlbjkljlkjsdfoiwje"

I then sourced that file, to export the variables on the CLI:

source .env

Finally, we are ready to issue the ACME challenge, and hopefully acme.sh will do the rest, including editing the DNS records:

acme.sh --issue --dns dns_dnsimple -d pi-hole.fotis.xyz

Once it is set up, acme.sh handles auto-renewal via a cron job. No more email snoozing, yay!

Installing the certificate

acme.sh will store the certificates in its internal folders (under /home/pi/.acme-sh). We are not supposed to rely on that structure, so it provides an --install-cert command to move the certificates to the correct place.

I used the following command

/home/pi/.acme.sh/acme.sh --install-cert -d pi-hole.fotis.xyz \
    --key-file       /home/pi/key.pem  \
    --fullchain-file /home/pi/cert.pem \
    --reloadcmd     "cat /home/pi/key.pem /home/pi/cert.pem > /home/pi/merged.pem && sudo service lighttpd restart"

Some things to note here. First of all, the home/pi/key.pem and /home/pi/cert.pem locations are meant to match the location where lighttpd expects them. I was confused by the terminology of acme.sh and of lighttpd, so I just named them in a way that made sense to me!

The “reloadcmd” argument is the tricky bit. It will run whenever the new certificate is installed, to prompt the server to use it. This command depends on your server. In my case, lighttpd needs the private key and chain file to be merged , so I do this as part of the setup script.

Hosts resolution

I edited the hosts file (/etc/hosts) on my own machine, to point the domain pi-hole.fotis.xyz to the known IP address of the pi-hole.

# Replace x.x with your local address :)
192.168.x.x pi-hole.fotis.xyz

I can now visit pi-hole.fotis.xyz locally, and see it secured, hooray!

Wrapping up

I hope this short guide on setting up HTTPS for local sites was useful! In addition to security, many modern browser APIs require HTTPS to function, so I suspect I will be using it again in the near future. Did I miss anything? Have you had a different way of doing this? Get in touch, and let’s talk!