Hugo PWA and Self Hosted Comments

Summary Process

1. Make a Hugo website
2. Add PWA configs
3. Get comments as JSON and add to data folder so Hugo can use during build time
4. NetlifyCMS saves images in one folder, at build time, copy the images to content folder and then process them with Hugo
5. Enjoy lightning speeds.

This article is long overdue. My brain right now is working at 200% trying to put together the development process involved - what I should have simplified using notes as I made the website. I will not make that mistake again in future.

Let me start.

Heart and Soul launched in 2019 powered by GatsbyJS. I loved Gatsby back then. I still do, and it runs Elevatika.

PWA and image processing attracted me to Gatsby. It has powerful plugins on that front.

However, I am not comfortable with the huge JSON file that is sent to the client just so routes can be hydrated.

As you launch your website, you won't notice it at first since you have just a few pages. And so when you visit Google Pagespeed and run your tests, the results will be excellent.

Wait until you have a few hundred articles. The site starts to drag. You begin to notice nonresponsiveness very fast. If you are on a fast connection, you will not notice. However, not all your visitors will be from the top cities in the world.

Another problem I noticed was by design. A website whose navigation is powered by JavaScript totally breaks when subjected to mobile browsers like Opera Mini.

It is not easy to tell people not to use Opera Mini when other websites just work. Breaks trust kind of.

Then the comments. The comments weren't loading as expected. Disqus can be slow. I use it on this blog, but just because comments are just not that important here. Users were complaining they just don't know what to do. How to comment that is. I recall Disqus UX used to be better some years ago. Not anymore. I had to find an alternative.

Welcome to Version 2.0

Gatsby is beautiful. How can I replicate the best features in Hugo? Almost everything I needed is there already, just that I needed some research. Let's see.

Image Processing

Hugo allows you to process images if they are page resources "bundled" together with the content. That means that you need to have the image within the content directory, not the static directory.

NetlifyCMS by design saves images in a separate directory from your markdown files. GatsbyJS did not care about this at all. It was able to process images no matter the source.

I could have just decided to put all the images in the content directory and Hugo would process them. But that is not ideal given that there are other images linked within the markdown files and those would be lost as NetlifyCMS uses the relative directory it uploaded the files to on links.

I needed a different approach.

I decided to write a script to copy the files to the content directory at build time. That way the images used in featured section would be processed and linked. As for those linked within the body of the markdown files, they would stay unprocessed. They are not that many though, only less than ten posts have the images in the body. The rest have only the featured image.

As you can probably guess, my approach means that if the content folder is big enough, it will be a problem since we are duplicating it. However, you can task a CI to do the job for you and you won't have any problem.

Some comments online suggest you can use symlinks. I didn't try.

The PWA

I found https://www.freecodecamp.org/news/build-a-pwa-from-scratch-with-html-css-and-javascript/ and it saved me a lot of time. Apparently, adding PWA is not that difficult. You just have to add a few JS files, a manifest.json file and then the necessary images.

Self Host Comments for a JAMstack Website

I believe HTML is capable enough and does not need much help submitting form data. I wanted the comment forms to be accessible even without JavaScript. Anyone should be able to submit without much trouble.

We have been slowly breaking the web by giving JavaScript all the power.

The approach I used is neat in my opinion.

I made a simple express API that receives the form post. No JS form submission. The API uses SQLite to store the comments.

By the way, I use Upcloud to host my Node.js projects using docker. I use Caddy server. You can click on this referral link to get $25 to try it out.

I could have also made a Lambda function or a Firebase Cloud Function for the same and use Firebase to store the comments. And I encourage anyone to go that route to avoid managing the API server. It's free.

When comments are submitted, they are queued for review. They appear after they have been approved.

The approval process is also simple, I made a simple way to load all the pending comments and the admin can simply change the status to approved.

The comments also have a hierarchy. Just a single level hierarchy though, for simplicity. So, you can reply to the comments but you cannot reply to the replies.

I made this possible by letting the comments have a parent. The following is a look at one of the comment objects:

{
    "date": "",
    "email": "",
    "id": 1,
    "comment": "",
    "name": "",
    "parent": "",
    "status": "",
    "url": ""
}

At build time, I download the comments using a node.js script that looks like so:

const fs = require('fs');
const fetch = require('node-fetch');
console.log("✔ Requesting comments from Elevatika Cloud...")
var url = 'https://example.com/comments/API_KEY';
fetch(url)
    .then(res => res.json()) //get json
    .then((comments) => {
        console.log("✔ Number of comments: ", Object.keys(comments).length); //not important
        console.log("✔ Writing data...");
        fs.writeFileSync('./data/comments_json.json', JSON.stringify(comments, null, 4), 'utf8', (err) => { //pretty json file
            if (err) throw err
        });
    })
    .catch(err => {
        console.log(err);
    });

Below is a dirty logic I use to display the comments (I don't know much Golang, so excuse my logic):

{{ $comments := .Site.Data.comments_json }}
{{ range $comments }}
    {{ $found := findRE $here .url }} //this matches the current url i.e current post
    {{ if and $found (not .parent)}} //if it is an original comment, not reply
    {{ $id := .id }}
    <div class="elev_single_comment" >
        <div class="elev_comment_name">{{ .name }}</div>
        <div class="elev_comment_content">{{ .comment | safeHTML }}</div>
        <form class="elev_hide_form" action="" method="POST">
            <fieldset>
                //the reply box with inputs
            </fieldset>
        </form>
        {{ range $comments }}
            {{ if eq $id .parent }} //display respective replies
            <div class="elev_reply">
                <div class="elev_comment_name">{{ .name }}</div>
                <div class="elev_comment_content">{{ .comment | safeHTML }}</div>
            </div>
            {{ end }}
        {{ end }}
    </div>
    <hr>
    {{ end }}
{{ end }}

Advantage of Adding Comments at Build Time

Your commenters are mentioning important keywords that you should be leveraging for SEO. Every time you use a JavaScript embedded comment section like Disqus, I am one of them, you are missing out.

The above Hugo code makes sure that at build time, the comments are added to the HTML therefore being like the good old serverside wordpress comments. Which are awesome.

The netlify.toml file allows you to specify custom build command. Therefore, my custom build command calls a build script that copies the images and then gets the comments from the API then invokes the hugo build command.

The netlify.toml:

[build]
  publish = "public"
  command = "bash build.sh"

[build.environment]
  HUGO_VERSION = "0.71.0"

The bash script:

#!/bin/bash
cp -Rf ./static/assets ./content/assets
node index.js
hugo

The Google PageSpeed Insight Results

Don't you love those speeds? The speed jumped from 79 to 98 and sometimes it is even 100%!

Feel free to leave any questions below.