How to implement HTTPS in Local Networks Using Let's Encrypt

Introduction

While browsing the internet, you may have noticed that URLs sometimes begin with "HTTPS" instead of "HTTP." This extra "S" signifies a secure connection, where your communication with the website is encrypted. This encryption ensures that only you and the website can decipher the exchanged information, protecting against "Man in the Middle" (MitM) attacks. In such attacks, an interceptor could eavesdrop or alter messages without detection. However, HTTPS scrambles the data into an indecipherable format for anyone intercepting it.

The foundation of HTTPS is the certificate authority (CA) system. CAs act as trusted validators on the internet, verifying website identities and issuing digital certificates, much like ID cards for websites. This system hinges on global recognition and trust in CAs. When a website presents a certificate signed by a known CA, your browser accepts it as a signal that the website is secure.

Yet, when it comes to securing local networks—such as those in your home or office—using global CAs presents challenges. These authorities cannot directly verify ownership of servers within private networks, which is crucial for issuing trusted certificates. An alternative could be setting up a Private CA, but this solution introduces its own complexities, such as the need for manual trust setup on new devices and maintaining security against vulnerabilities.

Furthermore, the threat of MitM attacks isn't limited to the vast internet; it's arguably more prevalent within local networks. Shared networks, like those in offices or public Wi-Fi spots, are fertile ground for malicious users to intercept unencrypted traffic. This risk underscores the necessity of HTTPS, ensuring communications are confidential and integrity is maintained, even in internal networks.

The approach we're about to explore, inspired by Joshua's method in securing web applications on private networks, involves using Certbot and Let's Encrypt for a streamlined solution. This method sidesteps direct server connection requirements by using DNS verification, making it suitable for internal networks. We begin by securing a domain name, setting up Certbot within Docker for certificate issuance, and finally configuring an Nginx web server to utilize the SSL/TLS certificate. This process, detailed further in the article, not only secures our network but does so in a cost-effective and accessible manner.

To facilitate the implementation of the HTTPS setup described in this article, I've prepared a comprehensive repository containing all the scripts. You can find everything you need to get started at https://github.com/ietxaniz/https-setup-local-networks.

Understanding HTTPS and Its Importance

HTTPS stands at the forefront of internet security, transforming the vulnerable HTTP into a secure channel that fiercely guards the data exchanged between your browser and websites. This transformation is powered by the SSL/TLS protocols, which encrypt data in transit, ensuring that your information remains confidential and intact. The cornerstone of this process is asymmetric encryption, where data encryption and decryption rely on two separate keys: a public key for encryption, accessible to anyone, and a private key for decryption, held securely by the server. This method ensures that even if the data is intercepted, without the private key, it remains unreadable.

The assurance of security doesn’t stop at encryption. When you visit a website using HTTPS, the presence of an SSL/TLS certificate acts as proof that you're indeed communicating with the intended server. This certificate, issued by trusted Certificate Authorities (CAs), is the result of a rigorous verification process that confirms the website's identity. It’s akin to verifying the authenticity of an ID card before granting access, protecting users from potential frauds and attacks.

At the beginning of each secure session, a handshake process takes place, laying the groundwork for a secure connection. This process includes:

• Server Authentication: The server presents its SSL/TLS certificate to the client, verifying its identity.
• Key Exchange: Utilizing asymmetric encryption, a session key is generated and shared securely between the client and the server.
• Secure Communication: With the session key established, the connection shifts to symmetric encryption, allowing for faster and secure data transfer.

This handshake is vital, not just for establishing encryption, but for ensuring that the communication channel is private, authenticated, and untampered with. It marks the transition from the public and private keys' computational rigor to the efficiency of symmetric encryption, maintaining security without the high computational load.

The significance of HTTPS transcends mere encryption. It signals to users that their connection is trustworthy, indicated by the padlock icon in their browser's address bar. This trust is crucial for websites handling sensitive information, as it assures users that their data is safe. Moreover, the need for periodic renewal of SSL/TLS certificates acts as a continual check, ensuring that encryption standards evolve to meet new security challenges and technological advances.

In the broader scope of the digital landscape, HTTPS is not just a technical specification; it is a foundational element of online security, privacy, and trust. Its role in safeguarding data integrity, authenticating website identities, and ensuring confidentiality is paramount. As we move towards securing local networks with HTTPS, understanding these core principles and processes is essential for implementing robust security measures that protect against the vulnerabilities of the digital age.

The Role of Certificate Authorities (CAs)

Certificate Authorities (CAs) play a pivotal role in the HTTPS ecosystem, acting as the backbone of web security by ensuring that communications between your web browser and websites are secure and trustworthy. CAs are organizations or entities trusted to issue digital certificates, which are essential components in the SSL/TLS protocol for establishing encrypted connections.

A digital certificate serves as an electronic "passport" for a website, proving its identity to visitors. When you access a HTTPS-secured website, your browser checks the website's digital certificate, verifying it was issued by a CA that the browser trusts. This verification process is crucial for preventing attackers from masquerading as legitimate websites, a common tactic in phishing attacks.

The distinction between global CAs and private CAs is essential for understanding their application in different network settings.

• Global CAs: These are widely recognized and trusted entities that issue certificates for websites across the internet. Examples include Let's Encrypt, VeriSign, and Comodo, among others. Certificates issued by global CAs are automatically trusted by most web browsers and operating systems, making them suitable for public-facing websites. However, their use in local networks can present challenges. Since global CAs verify the ownership of domain names as part of the certificate issuance process, they cannot directly validate servers that are not accessible on the public internet, such as those within a private corporate network.

• Private CAs: In contrast, private CAs are internal to an organization and not automatically trusted by external systems. They are used primarily to issue certificates for internal servers and applications, where external validation is not feasible or necessary. While private CAs offer the flexibility to secure internal communications, they come with the burden of managing trust manually. Devices and users within the network need to be configured to trust the private CA, a process that can be cumbersome and prone to errors if not handled carefully.

The primary challenge with private CAs is the management of trust. For a device or user to trust a certificate issued by a private CA, the CA's root certificate must be installed and trusted on that device. This requirement can complicate setups across large organizations with numerous devices and systems. Private CAs must be securely managed to prevent unauthorized access and issuance of certificates, which could lead to internal security breaches. The security of the CA's private key is paramount; if compromised, attackers could issue valid certificates for any domain, undermining the security of the entire network. Furthermore, deploying and maintaining a private CA requires ongoing management, including certificate renewal, revocation, and updating trust settings on devices. This overhead can be significant, particularly for large or dynamic environments.

In conclusion, the strategic use of global Certificate Authorities to issue certificates for servers in private networks represents a forward-thinking approach to network security. By adopting this method, organizations can harness the reliability and widespread trust of global CAs, sidestepping the complexities and potential security vulnerabilities inherent in managing private CAs. This approach not only simplifies the certification process but also enhances the security posture of private networks without compromising on trustworthiness or administrative overhead. As we move forward, leveraging global CAs in private networks emerges as a pragmatic solution, combining the best of both worlds—global trust and internal control—thereby eliminating the disadvantages traditionally associated with private Certificate Authorities.

Let's Encrypt and Certbot

Let's Encrypt represents a pivotal shift in web security, offering an automated and open service that provides digital certificates at no cost. Founded on the principle that achieving privacy and security on the internet should be accessible to all, Let's Encrypt simplifies the process of obtaining and renewing certificates. This initiative is crucial for small and large organizations alike, aiming to secure their local networks without incurring significant costs or administrative burdens.

The automation aspect of Let's Encrypt is particularly revolutionary. By eliminating the manual steps traditionally involved in applying for, renewing, and installing digital certificates, Let's Encrypt makes HTTPS adoption straightforward. This ease of access and maintenance encourages widespread adoption of HTTPS, making the internet a safer place for everyone.

Certbot, is a free, open-source software tool that automates the process of obtaining, installing, and renewing certificates from Let's Encrypt. Certbot is designed to work seamlessly with a wide range of web servers and operating systems, ensuring that secure HTTPS is attainable regardless of the technical environment.

In this tutorial, we leverage the Docker version of Certbot, a choice driven by Docker's ability to simplify infrastructure management. Using Docker, we avoid the need for complex installations, instead utilizing containers to maintain a clean and easily replicable setup. Our goal is to extend the base Certbot image to include curl, enabling us to handle more sophisticated scripts and interactions during the certificate setup process.

First, you'll need to create a Dockerfile. This file instructs Docker on how to build our customized Certbot image, incorporating additional tools like jq and curl for enhanced functionality:

FROM certbot/certbot:latest
RUN apk add --update --no-cache jq curl

To accompany the Dockerfile, a build script named build.sh automates the image creation process. This script is a simple Bash command that tells Docker to build our new image, tagging it as certbot-with-curl for easy reference:

#!/bin/bash
docker build -t certbot-with-curl .

To prepare these files, simply create them in a dedicated directory on your machine. For the Dockerfile, any text editor will suffice; just save the file with the name Dockerfile, without any extension. The build script should be saved as build.sh in the same directory. Ensure you grant execution permissions to the build script by running chmod +x build.sh in your terminal.

Executing the build script is straightforward. Open a terminal, navigate to the directory containing build.sh, and run the script by typing ./build.sh. This command initiates the Docker build process as defined in your Dockerfile, resulting in a new Docker image named certbot-with-curl that includes all necessary tools for our certificate management tasks.

With these preparations complete, we're set to dive into the certificate generation process, detailed in the following sections. This setup ensures we have a versatile and robust environment, leveraging Docker's encapsulation to streamline our HTTPS configuration tasks.

DNS Management

Securing your web application with HTTPS is crucial for protecting the data exchanged between your server and its clients. A key step in this process involves proving ownership of your domain, which is where DNS management becomes essential. This step is especially critical in environments with dynamic IP addresses or private networks. While there are various DNS providers available, this guide focuses on using DuckDNS. DuckDNS is a user-friendly, cost-effective option for linking your domain name to your server's IP address—even if it’s within a private network.

First, you'll need to create an account on DuckDNS. This platform is designed to simplify the domain management process and supports OAuth for account creation. This means you can quickly sign up using your existing Google, GitHub, or other social media accounts, eliminating the need for an additional set of credentials.

After creating your DuckDNS account, the next step is to set up your domain. In this guide, "inner-private" is used as an example domain name. Since "inner-private.duckdns.org" is already in use (for this example), you'll need to select an available name to proceed.

Follow these steps to configure your domain:

• Log into duckdns: after signing in, duckdns redirects to domain management page.
• Create a new domain: Enter your chosen domain name (e.g., "inner-private") in the designated field and click the "Add Domain" button.
• Configure IP address: DuckDNS will automatically detect and fill in your current public IP address. Since we’re mapping to a private address, manually change this to your specific private IP address (e.g., "192.168.1.103").
• Save changes: Ensure you save the modifications. DuckDNS will now link your domain to the specified private IP address, allowing network access under this domain name.

To verify domain ownership—critical for obtaining wildcard certificates—Let's Encrypt performs a DNS challenge. This requires adding a specific TXT record to your domain's DNS settings. Automating this process simplifies certificate renewal and issuance, particularly when using Certbot.

Below is the add_duckdns_txt_record.sh script, designed to update your domain's TXT record via DuckDNS's API. Note that we add a 90 seconds delay to allow some time for dns propagation:

#!/bin/sh

# Replace YOUR_TOKEN with your actual DuckDNS token
DUCKDNS_TOKEN="YOUR_TOKEN"
# Your DuckDNS domain
DOMAIN="inner-private"
# The value provided by Certbot
VALIDATION_STRING="$CERTBOT_VALIDATION"

# Update the TXT record
curl "https://www.duckdns.org/update?domains=_acme-challenge.$DOMAIN&token=$DUCKDNS_TOKEN&txt=$VALIDATION_STRING"
sleep 90

Before integrating this script with Certbot, let's ensure it works correctly:

Prepare the Script: Save the above script as add_duckdns_txt_record.sh. Replace "YOUR_TOKEN" with your actual DuckDNS token.

Make it Executable: Grant execution permissions to the script using chmod +x ./add_duckdns_txt_record.sh.

Test the Script:

Set a temporary value for the CERTBOT_VALIDATION variable:

export CERTBOT_VALIDATION="Example-token"

Run the script:

./add_duckdns_txt_record.sh

Verify the TXT record has been updated by querying your domain:

dig +short TXT _acme-challenge.inner-private.duckdns.org

You should see "Example-token" as the output, indicating the script has successfully updated the TXT record.

By following these steps, you’ll have configured your DuckDNS domain and prepared an automation script for the DNS challenge, streamlining the process for securing your web application with HTTPS. This setup not only simplifies certificate management but also ensures your domain's verification is handled efficiently.

Creating and Storing the Certificates

Storing SSL/TLS certificates in Docker volumes offers a convenient and secure way to manage these sensitive files. For this setup, we'll use two Docker volumes: https_keys for storing the keys and https_challenge for storing temporary files used by Certbot during the certificate issuance process. Follow the steps below to generate and renew your certificates effectively.

Before generating the certificates, we need to create the necessary Docker volumes. This ensures that our certificates and keys are stored persistently and securely, separate from the container's ephemeral filesystem. Use the following script to create or renew the Docker volumes:

# create_volumes.sh
#!/bin/bash
# Remove existing volumes (if necessary)
docker volume rm https_keys https_challenge
# Create new volumes for storing certificates and challenges
docker volume create https_keys
docker volume create https_challenge

Before running this script, please ensure that removing the existing volumes is acceptable, as this action will delete any stored certificates and keys. This step is generally only needed when setting up for the first time or intentionally resetting your certificates.

To generate a certificate, we leverage a customized Certbot Docker container. This container interacts with Let's Encrypt to automate the certificate issuance process, including domain verification and certificate renewal. The script below initiates the certificate generation process, including hooks for DNS challenges which are required for domain verification. Take into account that the script considers that the script from previous section add_duckdns_txt_record.sh is stored in ./script subfolder:

# create_certificates.sh
#!/bin/bash
# Define your domain
DOMAIN="inner-private.duckdns.org"
# Run Certbot in Docker
docker run -it --rm \
    -v https_keys:/etc/letsencrypt \
    -v https_challenge:/var/lib/letsencrypt \
    -v "$(pwd)/scripts:/scripts" \
    --env CERTBOT_DOMAIN="$DOMAIN" \
    certbot-with-curl \
    certonly \
    --staging \
    --verbose \
    --manual \
    --preferred-challenges dns \
    --manual-auth-hook "/scripts/add_duckdns_txt_record.sh $DOMAIN" \
    --manual-cleanup-hook "/scripts/remove_duckdns_txt_record.sh $DOMAIN" \
    -d "$DOMAIN" \
    --non-interactive \
    --agree-tos \
    --email me@inner-private.duckdns.org

After verifying that your setup works correctly in the staging environment, prepare to issue a production certificate. Remove the --staging flag from the create_certificates.sh script and re-run it. This action requests a production-ready certificate from Let's Encrypt, subject to rate limits and stricter validation.

Let's Encrypt certificates are valid for 90 days. To renew your certificate, simply re-execute the create_certificates.sh script. It's recommended to automate this process (for example, by setting up a cron job) to ensure your certificates remain valid without manual intervention.

This approach to certificate management with Docker and Certbot simplifies the process of securing your applications. By automating certificate issuance and renewal, you can ensure continuous protection for your domains with minimal overhead. Always test changes in the staging environment before applying them to production to avoid disruptions and ensure a smooth deployment.

Testing everything with nginx

After setting up HTTPS using Certbot and Let’s Encrypt, the next step is to deploy a web server to serve your content securely over HTTPS. This section provides an example of hosting a simple web server within an internal network, utilizing an Nginx container and a Let's Encrypt signed certificate. While this example is straightforward, it lays the groundwork for more complex configurations, such as setting up Nginx as a reverse proxy for more featured services.

First, we need a simple web page to serve. Create a directory named after your domain (in this case, inner-private.duckdns.org) and inside that directory, create an index.html file. This file will be our sample web page:

<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8">
  <meta name="viewport" content="width=device-width, initial-scale=1.0">
  <title>Internal</title>
</head>
<body>
  <h1>Internal web service with HTTPS.</h1>
</body>
</html>

This simple HTML page will serve as the content we'll protect with HTTPS.

Next, we need to configure Nginx to serve our website securely and redirect HTTP traffic to HTTPS. Create an Nginx configuration file with the name corresponding to your domain, for example, inner-private.duckdns.org.conf, and populate it with the following configuration:

server {
  listen 80;
  server_name inner-private.duckdns.org;
  location / {
    return 301 https://$host$request_uri;
  }
}

server {
  listen 443 ssl;
  server_name inner-private.duckdns.org;

  ssl_certificate /etc/letsencrypt/live/inner-private.duckdns.org/fullchain.pem;
  ssl_certificate_key /etc/letsencrypt/live/inner-private.duckdns.org/privkey.pem;

  location / {
    root /var/www/inner-private.duckdns.org;
    index index.html index.htm;
  }

  # Additional SSL settings (optional but recommended)
  ssl_protocols TLSv1.2 TLSv1.3;
  ssl_prefer_server_ciphers on;
  ssl_ciphers "ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384";
  ssl_session_cache shared:SSL:1m;
  ssl_session_timeout 10m;
}

This configuration does two main things:

• Redirects all HTTP traffic to HTTPS, ensuring secure communication.
• Serves your HTML page over HTTPS using the SSL/TLS certificate generated by Let’s Encrypt.

Finally, use Docker to run Nginx and serve your site. The script below sets up and starts the Nginx server in a container, mounting the necessary volumes for the SSL certificates, website content, and Nginx configuration:

#!/bin/bash
docker run \
  --name rev-proxy \
  -p 80:80 \
  -p 443:443 \
  -v https_keys:/etc/letsencrypt \
  -v https_challenge:/var/lib/letsencrypt \
  -v $(pwd)/nginx:/etc/nginx/conf.d \
  -v $(pwd)/inner-private.duckdns.org:/var/www/inner-private.duckdns.org \
  nginx:1.25.1-alpine3.17-slim

This script starts an Nginx container with the following configurations:

• Ports: Maps port 80 (HTTP) and 443 (HTTPS) from the container to your host, making your site accessible.
• Volumes: Mounts the necessary directories for SSL certificates, Nginx configuration, and your website content.

After running this script, your internal web service should be accessible over HTTPS, showcasing a secure setup. This example serves as a foundation, which you can expand upon based on your requirements, such as configuring Nginx as a reverse proxy for more complex applications.

Conclusion

This article showcases an example of how to combine Certbot, Let’s Encrypt, and Docker within an Nginx setup to secure web applications on private networks. By delving into the significance of HTTPS and outlining a step-by-step implementation process, it aims to equip readers with the knowledge and tools necessary to enhance their network's security. The detailed guide leverages the principles of automation and the reliability of trusted global Certificate Authorities, making advanced security measures more accessible and manageable.

Given the dynamic nature of cybersecurity, there is always the possibility for improvement and the emergence of new challenges. The collective wisdom and experiences of the community are invaluable for identifying potential oversights or areas where security practices could be optimized. Feedback and contributions from peers are encouraged to refine and advance the methodologies presented.

If any inaccuracies are noticed or if there are suggestions for enhancing the security aspects discussed, the community's feedback would be greatly appreciated.