DEV Community

TechSynthesis
TechSynthesis

Posted on

Automagically creating a Sitemap for React, Node.js, MongoDB website

#react #node #tutorial #javascript

Sitemaps play an important role in SEO optimisation. Thus having a current sitemap is important. Automating a sitemap is easy. My sitemap generator runs on my node.js web server, automatically updating my sitemap.xml in my create-react-app website, public folder when I create or delete a page from site.

Step 1. Create a new generate-sitemap.js file

Step 2-A. Import the following 

import { writeFileSync } from "fs";
import mongoose from "mongoose";

//Import the model which you want to autogenerate. 
import Product from "../models/productModel.js";
Enter fullscreen mode Exit fullscreen mode

Step 2-B. Connect to your DB 

const mongodbUrl = config.MONGODB_URL;
mongoose
    .connect(mongodbUrl, {
        useNewUrlParser: true,
        useUnifiedTopology: true,
        useCreateIndex: true,
    })
    .catch((error) => console.log(error.reason));
Enter fullscreen mode Exit fullscreen mode

Step 2-C. Create the sitemap generator function 

export const generateSitemap = async () => {
    const products = await Product.find();
    const sitemap = ``;
}
Enter fullscreen mode Exit fullscreen mode

Step 2-D. Add the static pages

(non-changing urls i.e About, Home, Contact, Search, TOS, Privacy Policy, etc). 

const generateSitemap = async () => {
    const products = await Product.find();
    const sitemap = `
    <?xml version="1.0" encoding="UTF-8"?>
    <urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
    <url>
        <loc>https://www.yourWebsiteHere.com/</loc>
    </url>
    <url>
        <loc>https://www.yourWebsiteHere.com/about</loc>
    </url>
    <url>
        <loc>https://www.yourWebsiteHere.com/contact</loc>
    </url>
    <url>
        <loc>https://www.yourWebsiteHere.com/search</loc>
    </url>
    <url>
        <loc>https://www.yourWebsiteHere.com/rules/privacy</loc>
    </url>
    <url>
        <loc>https://www.yourWebsiteHere.com/rules/terms-of-use</loc>
    </url>
    <url>
        <loc>https://www.yourWebsiteHere.com/rules/shipping-and-returns</loc>
    </url>
</urlset>
`;
}

export default generateSitemap
Enter fullscreen mode Exit fullscreen mode

Step 2-E. Add the dynamic pages after the last </url>

It can be products, categories, articles, etc. 

  ${products
            .map((page) => {
                return `
              <url>
                  <loc>${`https://www.yourWebsiteHere.com/product/${page.slug}`}</loc>
              </url>
            `;
            })
            .join("")}
Enter fullscreen mode Exit fullscreen mode

Step 2-F: Write the file to your desired location.

I'm saving it to my public folder. 

writeFileSync("./frontend/public/sitemap.xml", sitemap);
console.log("genarated Sitemap successfully");
    return true;
Enter fullscreen mode Exit fullscreen mode

Step 3. Recreate the sitemap every time an new product is added.

I want the generateSitemap function to run every time I add a new product. So I'm calling it after my product is created.

import generateSitemap from "../scripts/generate-sitemap.js";

productRouter.post(
    "/",
    isAuth,
    expressAsyncHandler(async (req, res) => {
            const createdProduct = await product.save();
            //calling generateSitemap
            generateSitemap();
            res.send({
                message: "Product has been added successfully!",
                product: createdProduct,
            });

    });
Enter fullscreen mode Exit fullscreen mode

You can do something similar or create a cron job to recreate the sitemap at a set interval.

I'm using node-cron for that. I've scheduled it to run every morning at 6am and I've set the timezone closest to mine.

Install node-cron using npm install --save node-cron

import cron from "node-cron";
cron.schedule(
    "0 6 * * *",
    () => {
        generateSitemap();
    },
    {
        scheduled: true,
        timezone: "Asia/Colombo",
    },
);
Enter fullscreen mode Exit fullscreen mode

Finally once your site map is generated you can access it by typing your websitename.com/sitemap.xml or if you're on localhost then localhost:port/sitemap.xml.

Complete Code for generate-sitemap.js

import { writeFileSync } from "fs";
import mongoose from "mongoose";
import config from "../config.js";

import Product from "../models/productModel.js";

const mongodbUrl = config.MONGODB_URL;
mongoose
    .connect(mongodbUrl, {
        useNewUrlParser: true,
        useUnifiedTopology: true,
        useCreateIndex: true,
    })
    .catch((error) => console.log(error.reason));

const generateSitemap = async () => {
    const products = await Product.find();
    const sitemap = `
    <?xml version="1.0" encoding="UTF-8"?>
    <urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
    <url>
        <loc>https://www.yourWebsiteHere.com/</loc>
    </url>
    <url>
        <loc>https://www.yourWebsiteHere.com/about</loc>
    </url>
    <url>
        <loc>https://www.yourWebsiteHere.com/contact</loc>
    </url>
    <url>
        <loc>https://www.yourWebsiteHere.com/search</loc>
    </url>
    <url>
        <loc>https://www.yourWebsiteHere.com/rules/privacy</loc>
    </url>
    <url>
        <loc>https://www.yourWebsiteHere.com/rules/terms-of-use</loc>
    </url>
    <url>
        <loc>https://www.yourWebsiteHere.com/rules/shipping-and-returns</loc>
    </url>
        ${products
            .map((page) => {
                return `
              <url>
                  <loc>${`https://www.yourWebsiteHere.com/product/${page.slug}`}</loc>
              </url>
            `;
            })
            .join("")}
    </urlset>
    `;

    writeFileSync("./frontend/public/sitemap.xml", sitemap);
    console.log("genarated Sitemap successfully");
    return;
};

export default generateSitemap;
Enter fullscreen mode Exit fullscreen mode

Top comments (2)

Collapse
 
techsynthesis profile image
TechSynthesis •

If you have issues with the post or if you get stuck at any point do let me know. This is my first time documenting something like this.

Collapse
 
dragos99 profile image
Dragos George •

Thanks, this is amazing!

Read next

srishtikprasad profile image

Unlocking Performance: The Power of Caching with Redis

Srishti Prasad -

rivea0 profile image

LeetCode Meditations: Serialize and Deserialize Binary Tree

Eda -

kartikmehta8 profile image

Exploring Graph Databases: Neo4j

Kartik Mehta -

0xmesto profile image

Unofficial ChatGPT API with simple agent-based

0xMesto -