Make A Hugo Static Blog Inside A Django App

I have a side project and I'd like to do some content marketing to potential customers to show how my product is useful. To do this, I need a blog for my project.

Maybe you need a blog for your project too. Have you thought about where your blog will exist on the internet? For me, I considered two choices:

1. Use a subdomain like blog.mysite.com.
2. Use a route style like mysite.com/blog/. (I learned in my research that SEO experts call this a "subfolder" style.)

I chose the latter approach.

I'm also a big fan of statically generated websites. I like to keep my articles in version control and write my content in Markdown. This approach works well for my writing flow. In fact, if you're reading this on my website, mattlayman.com, you're reading content that was generated by Hugo, my static site generator of choice.

Since my Django application runs on my main domain, how could I include a route-based blog onto the domain without tripping over the app?

I could see two strategies to making a route-based blog work.

1. Put some software between browsers and my application server that can intercept and route blog traffic to the static files generated by Hugo.
2. Make the application server serve the blog.

Putting Software In Front Of The App Server

This is a well-trodden path. This is also the path that I didn't pick (more on that later).

In many (most?) Django deployments, the Django application runs with a Python application server like Gunicorn or uWSGI. These application servers have the job of delegating the dynamic requests to Django views.

What do we do with requests that aren't dynamic like JavaScript or image files? Django has a process to manage these static files that will collect all the files into a single directory that can be served by other software.

What other software? Typically, that other software is a general purpose web server like Nginx or the Apache HTTP Server. When one of these web servers is in between and delegating to the Python application server, we call the web server a reverse proxy. Check out Cloudflare's reverse proxy article for a good explanation of why this kind of server is considered a reverse proxy.

A web server like Nginx can be configured to serve static files at a particular route. For Django's typical static file handling, you would configure Nginx to route the static files directory that Django produces to something like /static/. For any request for a static file that comes to your site, Nginx would detect that the path starts with /static/ and would send back the file directly rather than requesting it from the application server.

Knowing that, it's not much of a conceptual leap to see how to serve a static blog. In this scenario, you would generate the blog with your tool of choice and configure Nginx to route anything coming to /blog/ to the output of your static site generator.

To be frank, if your infrastructure can handle this style well, this will probably be an easier approach. If you're feeling a bit more adventurous or like walking a different path, read on.

The Road Less Traveled

Another way of serving static files for your Django app is to let the application server do it. This approach is not as performant as the reverse proxy approach, but it has the advantage of being a simpler setup because you only have one kind of server running, not two.

When you want to use this style, you'd reach for WhiteNoise.

For Django projects, WhiteNoise is designed to work with Django's static files scheme. That means that the library will have no trouble serving your CSS, JS, images, or whatever else. This also means that you can expect all of these files to be served out of /static/.

If you're ok with serving your blog from /static/blog/, then your life would be pretty simple. When you deploy, you'd generate your blog content from your static site generator, include the output directory as a directory in the STATICFILES_DIRS Django setting, and you're done.

That kind of URL path sounds gross to me. What casual non-tech reader would expect to read a blog post at /static/blog/? Yuck. That kind of reader is unlikely to know what "static" would mean.

My goal was to get WhiteNoise to serve my blog at /blog/. That's the setup. Let's see how it worked out.

The Details

Before seeing all the details, let me make sure I address why I did this.

My application is running on Heroku. Heroku makes deployment so simple for basic apps. With a small file called a Procfile that looks like

release: python manage.py migrate
web: gunicorn project.wsgi --log-file -

I can get an entirely operational application on their platform. The downside of using a Platform as a Service (PaaS) like Heroku is that I have less control of the environment.

In this circumstance, I don't have the ability to introduce a reverse proxy like Nginx. I could cobble together some scheme with a shell script that would let me run both Nginx and Gunicorn, but I'd have difficulty guaranteeing that both processes would stay running.

Unless I wanted the blog to run on a separate subdomain, which I already mentioned that I don't, I need to make the Gunicorn application server serve the blog.

First, let's get the blog itself going. I'm going to skip most of the details of working with Hugo, but I want you to have some names that you can consider as we work through this problem. From the root of my repository, I ran:

$ hugo new site blog

This created a new Hugo site with some empty directories. Because I wanted to keep all of the directories, I added some hidden files so that Git would track them. For example,

$ touch blog/data/.gitkeep

Later in my experimentation, I found that the config.toml needed to be in the root of the repository. Since I didn't want to fill the repository root with other Hugo directories, I had to adjust some variables in the config file to look in the blog directory.

# config.toml

archetypeDir = "blog/archetypes"
contentDir = "blog/content"
data = "blog/data"
layoutDir = "blog/layouts"
staticDir = "blog/static"
themesDir = "blog/themes"

I also set the directory where I wanted the blog output. This name is important later.

# config.toml

publishDir = "blog_out"

To finish off the Hugo setup (aside from the actual blog content generation which I'm not going to describe), I added Hugo generated directories to my .gitignore.

# Blog
blog_out/
resources/

With this much configuration, I can generate my product's blog with a single command.

$ hugo

                   | EN
+------------------+----+
  Pages            | 13
  Paginator pages  |  0
  Non-page files   |  2
  Static files     | 10
  Processed images |  7
  Aliases          |  0
  Sitemaps         |  1
  Cleaned          |  0

Total in 14 ms

14ms! It's so fast!

My next job was to teach Heroku how to generate the blog with each deployment. Heroku tries to figure out how to build your application by checking for certain files. Because of the manage.py file, Heroku detected that I have a Python project.

When there are multiple types of things to build, you have to be a bit more explicit. Thus, I needed to add a buildpack. Buildpacks are responsible for assembling a project's code into a format that the Heroku platform can run. To make Hugo go, I added the roperzh/hugo community contributed buildpack.

$ heroku buildpacks:add --index 1 roperzh/hugo

I set my Hugo version in Heroku to the same version that I use locally on my Mac to ensure consistency.

$ heroku config:set HUGO_VERSION=0.46

From this configuration, my Heroku deployments now build my Hugo blog and store the output content in the blog_out directory of the built application artifact (which Heroku calls a "slug").

We can take another step to optimize the blog. WhiteNoise will serve a compressed version (either gzip or brotli) if there is a file on disk with the same name and ending with a .gz or .br extension (e.g., index.html and index.html.gz).

Once Hugo is done generating the blog output, we can instruct WhiteNoise to compress the files. To do this, I used a bin/post_compile script that runs as part of the Python buildpack. My script looks like:

#!/bin/bash

set -e

python -m whitenoise.compress blog_out

This generates the compressed versions so that my application server can serve fewer bytes when sending static blog files to a browser.

Now that the static content side is ready to go, let's teach Django how to serve the blog.

In Django, WhiteNoise works by running a Django middleware. This middleware is designed to run very early in the stack of middleware to intercept requests to static files and return them before the application server wastes too much time processing the request.

The problem with the middlware (if I can even call it a problem) is that it is designed to work exclusively with the static files mechanism that Django exposes as the static files interface. This means that it serves content for /static/ URLs from directories that are either in static directories inside of each Django app or static files included in STATICFILES_DIRS.

Unfortunately, there's no Django setting that will permit a developer to say "Hey, WhiteNoise, please serve these files at this other path too!"

My solution to this dilemma was to "use the source, Luke (uh, Matt)!" You can see in Using WhiteNoise with any WSGI application that the WhiteNoise class has a method named add_files. This method takes a directory and serves those files at some developer-defined prefix. While inspecting the source, I found that the WhiteNoiseMiddleware is a subclass of WhiteNoise.

Enter MoreWhiteNoiseMiddleware.

I decided to make a subclass of the WhiteNoiseMiddleware to take advantage of the add_files method for my Django project. (My side project is called "homeschool" so you'll see that in the code snippets below.)

# homeschool/middlware.py

from django.conf import settings
from whitenoise.middleware import WhiteNoiseMiddleware

class MoreWhiteNoiseMiddleware(WhiteNoiseMiddleware):
    def __init__(self, get_response=None, settings=settings):
        super().__init__(get_response, settings=settings)
        for more_noise in settings.MORE_WHITENOISE:
            self.add_files(
                more_noise["directory"], prefix=more_noise["prefix"])

Then I made the following changes to my settings file.

# project/settings.py

MIDDLEWARE = [
    "django.middleware.security.SecurityMiddleware",
    "homeschool.middleware.MoreWhiteNoiseMiddleware",
    ...
]

...

MORE_WHITENOISE = [
    {"directory": os.path.join(BASE_DIR, "blog_out"), "prefix": "blog/"}
]

WHITENOISE_INDEX_FILE = True

That last setting will make sure that WhiteNoise serves any directory with an index.html as the file. That means that my blog post content that Hugo puts into /blog/some-post/index.html will be accessible at mysite.com/blog/some-post/. Other rules about this setting are in the documentation.

With this tiny middleware subclass, I can serve up more static directories! My product blog is served up at the pretty /blog/ URLs that I wanted. Victory!

I'm only doing this for my product's blog now, but I will probably do something similar with the product documentation in the future.

Tradeoffs

What's the catch? It seems like there's always a catch.

The biggest catch is caching, or, the lack thereof. With the scheme I've described, Django isn't able to generate filenames for the blog files that include the hash of the file content.

In a normal static files setup, you can use ManifestStaticFilesStorage to generate those file names. With that storage engine, Django will generate a manifest file that stores a dictionary of original filenames to the versioned filename that includes the hash. Django uses the manifest during template rendering to serve up HTML content that includes the hashes (e.g., a file named base.css would be sent to the user as base.1234abcd.css).

Because Django sends out the versioned filenames for browsers, when the browser comes back to the server to request a CSS file like base.1234abcd.css, WhiteNoise can detect that the file is "versioned." With a versioned file, WhiteNoise will set the Cache-Control HTTP cache header to tell the browser that the file can be safely cached for a very long time.

The content generated by Hugo doesn't go through the Django template engine and won't include those version hashes. Thus, WhiteNoise can't detect the files won't change because of the absense of hashes. Since the code doesn't know if the file will change, it can't set Cache-Control far into the future. Instead, it will set the header to one minute which is configurable via the WHITENOISE_MAX_AGE setting.

For a small product like mine, this tradeoff is totally reasonable. If I had a product blog with massive amounts of traffic, I'd probably have a more complex infrastructure anyway and be in a position to use a reverse proxy instead.

The other minor tradeoff is that WHITENOISE_INDEX_FILE setting. By enabling that, I open up my Django server to serving directories that include index.html. This is good and desirable for the blog, but the side effect is that any other directory in my static files that happens to have an index.html file in it is also now available. That may not affect your app if you try this approach, but it's something to be cognizant of.

Summary

I started this adventure by looking for an alternative way to serve a blog for my Heroku project.

In the process, we learned about:

• Heroku buildpacks and how to use multiple buildpacks to generate an app that requires multiple tools
• WhiteNoise and how to customize the middleware to give it more files to serve
• Caching and the tradeoffs associated with my approach

I hope you found this little adventure interesting. Next time you need some static content for your Django project outside of /static/, now you know of an option that doesn't include subdomains or a reverse proxy!

If you have questions or enjoyed this article, please feel free to message me on Twitter at @mblayman or share if you think others might be interested too.