Dynamic Rendering, A Simple Solution for SEO Problems on Single-page Application Website

When making a website, the goal of your website is to be visited by the user/customer, right? There are various ways for websites can be visited. Yep, one of them is using the SEO technique, this is how to make your website easily found only through search engines like Google, Bing, or Duckduckgo.

Everything will be okay until you realize that your website is built using Javascript and most of the content is generated by Javascript. But calm down, search engines like Google are now more advanced in reading Javascript tho. Since May 2019 Google has been using the evergreen more about it can be read here, they claim Google's latest Evergreen bot can be more reliable in rendering Javascript content, the latest Googlebot now uses Chrome version 74 which has the same capabilities as your Chrome browser in rendering Javascript.

Yep, it's Google, then what if you share your website on social media? What about Facebook's crawlers or Twitter's crawlers?

If you know, it's not only Google, Bing, or Duckduckgo who have crawlers, social media like Facebook and Twitter also have crawlers that are purposed to get meta and display it into an object from a website that is shared to social media.

How to do this?

Facebook and Twitter have their own tags so that their bot can detect and create data objects to display as shown above.

<!-- Open Graph / Facebook -->
<meta property="og:type" content="website">
<meta property="og:url" content="https://example.io/">
<meta property="og:title" content="Lorem ipsum dolor sit amet, consectetur adipiscing elit">
<meta property="og:description" content="Lorem ipsum dolor sit amet, consectetur adipiscing elit">
<meta property="og:image" content="https://example.io/assets/meta-tag.png">

<!-- Twitter -->
<meta property="twitter:card" content="summary_large_image">
<meta property="twitter:url" content="https://example.io/">
<meta property="twitter:title" content="Lorem ipsum dolor sit amet, consectetur adipiscing elit">
<meta property="twitter:description" content="Lorem ipsum dolor sit amet, consectetur adipiscing elit">
<meta property="twitter:image" content="https://example.io/assets/meta-tags.png">

BUT, when your website is a single-page application, then you need to prepare when Facebook or Twitter bot cannot read the meta tags or contents on your website. Based on my experiment, I did when this article was written in May 2020, Facebook bot is not capable of reading SPA or websites that its content is generated by Javascript. Pathetic.

Then, how?

Dynamic Rendering can be your best friend, although there are other ways, like converting your website into a static site.

OK, let's say you only want SPA, and all we need to do is dynamic rendering.

So what is Dynamic Rendering?

If we look at its name 'dynamic', don't forget if you use dynamic rendering then you need a server, in my case I use a NodeJS server. With dynamic rendering, the web pages that will be delivered by the server are different depending on the detected user-agent. If its detected user agent is a bot, the web page that will be delivered to the client is a static generated of the requested page because before sending it to the client, Puppeteer will process the webpage and render it first. But if the detected user is a real human, then the page that will be sent to the client is html, js, and css and will be rendered right in the browser of the user.

How can we implement it?

First, you need a server that can support NodeJS, if you don't have it you can use Heroku.

The easy way is to create your project folder and then do npm init.

Then install several packages as below:

ExpressJS: npm install express

Puppeteer: npm install puppeteer

Useragent: npm install useragent

After all of the three packages are installed, you will need to create the file index.js as an entry point on your server side.

//index.js

const express = require('express');
const puppeteer = require('puppeteer');
const ua = require('useragent');
const app = express();
var path = require("path");

const directory = 'dist';
const dist = path.join(__dirname, directory)

const port = process.env.PORT || 3000;

//you can put your puppeteer middleware here later


app.use('*', (req, res) => {
    res.sendFile(path.join(dist, 'index.html'));
})

app.listen(port, () => {
    console.log(`Web server is running at port ${port}`);
});

Add this code to middleware to detect user agents.

function isBot (useragent) {
    const agent = ua.is(useragent);
    return !agent.webkit && !agent.opera && !agent.ie &&
        !agent.chrome && !agent.safari && !agent.mobile_safari &&
        !agent.firefox && !agent.mozilla && !agent.android;
}

const uAgentMiddleware = async (req, res, next) => {
    const local_url = 'YOUR_BASE_URL'

    if (!isBot(req.headers['user-agent'])) {
        next ()
    } else {

        try {
            const browser = await puppeteer.launch({
              'args' : [
                '--no-sandbox',
                '--disable-setuid-sandbox'
              ]
            })
            const page = await browser.newPage();
            await page.setUserAgent('Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2228.0 Safari/537.36');
            await page.goto(local_url, {
                waitUntil: "networkidle0",
            });
            const html = await page.evaluate(() => {
                return document.documentElement.innerHTML;
            });
            await browser.close();

            res.send(html);
        } catch (err) {
            res.send(err)
        }
    }
}

app.use(uAgentMiddleware)

After adding the code above, then make sure you have copied your dist folder or the Vue build folder (in this case I use VueJS) to the same folder as index.js.

Finally in package.json add a script like the following to runindex.js.

Then just run with npm run start to start the server.

//package.json

//....  
"scripts": {
  "start": "node index.js"
},
//...