Multi-app web server in 30 terminal commands

"Spices and Stuff" by Benson Kua is licensed with CC BY-NC-SA 2.0.

In a short while you'll be able to set up your own server for hosting multiple web applications from scratch. The server will:

• Serve a static website and a Python application using Docker
• Follow best practices of web security and server management
• Be easy to reinstall and develop further with e.g. Node or a database
• Keep you in control of everything

Estimates to:

• read: 10 min
• follow along: 30 min to several hours depending on experience level and potential issues, ~$10 / month for domain and server

This tutorial is aimed at you if you want to elegantly manage something more than a static website, like a full web application, your own cloud storage, a game server etc. or a combination of all of them! Or if you're just interested in figuring out cool stuff.

We will go fast, so don't feel bad if you have to pause, think and search for information. This took me a long while to figure out too. I don't aim to explain everything, but instead provide a good template to start building on.

Prerequisites

• A domain name and control of the DNS records, most likely on your provider's website.
• A Debian server with root access. A virtual private server is probably your best bet.
• A local Unix machine (Linux / Mac) with common tools like git, ssh and scp installed. On Windows, you can use Windows Subsystem for Linux.

Setting up a domain and a server should cost approximately $5 to $20 per month long-term, depending on the domain and caliber of the server. For this tutorial, even as little as 10 GB of disk space and 2 GB of RAM should be safe. Processor power won't be an issue.

DNS configuration

To point the abstract domain to your physical server, we'll set up two DNS records. A root address record and a wildcard for any subdomains. If you want to be strict, you can also set each subdomain separately. Substitute the IP address below with your server. If the address wasn't provided to you, you can determine it inside your server.

@ 1800 IN A 1.1.1.1
* 1800 IN A 1.1.1.1

After configuring DNS you should be able to resolve your domain. Propagating the records could take hours, so to avoid checking constantly we'll watch the status. Continue the tutorial when the address is resolved successfully. Note that the request will still result in error, but the server IP should be found.

curl -v DOMAIN              # check if domain can be resolved
watch -n 10 curl -v DOMAIN  # check every 10s

Template repository

Unfortunately there's no way to keep this tutorial short without some help. We'll use a template repository that contains build scripts, configuration and dummy website content. Consider forking the repository or using it as a template to manage your own future project!

git clone https://github.com/felix-hilden/server-template.git
cd server-template

Server configuration

Let's get to the meat of the issue and start configuring our server. The setup steps below are laid out on two indentation levels, the first being on your local machine, and the second with a remote connection to the server. Please read about the commands being used and look at the scripts that are executed to understand what is being done.

First, we avoid root access by creating a new admin user on the server. Please substitute DOMAIN for your domain name and USER for your desired user name. The commands below assume you have SSH access immediately, but if you're using other login methods you can run the commands in user-setup manually.

cd server

scp user-setup root@DOMAIN:~   # copy setup script to server
ssh root@DOMAIN                # login as root
    cat user-setup             # peek at the script
    chmod +x user-setup        # make it executable
    ./user-setup USER          # setup new sudo user
    rm user-setup              # remove setup script

The next step will configure SSH and a firewall along with a few important applications for our system. Before closing the root connection, continue the setup in another terminal or make sure you can access the server another way if something goes wrong and you are logged out without a way to enter. The configuration will disable root access and password logins through SSH for security.

cat setup-ssh            # peek at the script
chmod +x setup-ssh       # make it executable
./setup-ssh USER DOMAIN  # generate and send SSH keys

scp files/* DOMAIN:~     # copy build files to server
ssh DOMAIN               # login as new user
    cat build            # peek at the script
    chmod +x build       # make it executable
    ./build USER         # build server
    rm *                 # remove build files

For additional configuration, this nixCraft article has great tips on Linux security and it seems to be kept up to date: https://www.cyberciti.biz/tips/linux-security.html

Applications

From now on, we'll only be working inside of the server. So let's clone the template repository on the server too. Before setting up a proper deployment process, the git project will do fine for synchronising changes across our local machine and the server.

git clone https://github.com/felix-hilden/server-template.git
cd server-template

At this point it is appropriate to explain the basics of our tech stack.

• Docker: provides light virtual machines for us to isolate services with. Instead of configuring our host server further, which quickly becomes unmanageable, we'll build and run containers defined in our git project, where all the relevant information is stored in one place.
• nginx: a staple web server. We'll use it for both serving a static website in its container and as a proxy to route traffic to our Python application in another container.
• Python: using the trending FastAPI framework, we'll serve a simple site with an API.

To get the website itself up and running, we'll only need to follow a few steps. First, replace all occurences of "example.com" in the template files with your domain using the magic script below. Trust me. Or this guy.

# Linux
grep -RIl 'example.com' | xargs sed -i 's/example.com/DOMAIN/g'

# Mac OS
grep -RIl 'example.com' | xargs sed -i '' 's/example.com/DOMAIN/g'

git diff  # inspect the result

Then, we'll issue SSL certificates for your domain to be able to use secure connections.

make certify  # bring up a dummy website to issue certificates

Assuming that the certificates were issued successfully, we can start up the real applications.

make up-prod  # serve applications on HTTPS
make logs     # start following logs to see if initialisation was successful

Behind the scenes our Makefile is orchestrating a set of containers to run while setting up networking and shared filesystems to find our configuration. We should now be able to visit the domain in an ordinary browser! Or get the home page with curl.

curl DOMAIN      # static site
curl app.DOMAIN  # Python app

Developing the site

The repository includes configuration to run the site locally to facilitate development. On your local machine, install Docker Desktop and run:

make  # bring up development website & inspect logs
curl localhost  # get home page

You can now modify the contents of website/site and app/src to start building your applications. The sites will be updated with your latest modifications while they're running. If something goes wrong, the default make target can be used to restart.

Farewell

I hope you've found this terse tutorial useful. Thank you for reading. The template is lacking in many ways, particularly when it comes to actual software development. However, we won't set up a full development environment for the website or Python here. That, along with automating the release process, is left as an exercise to you, my dear reader.