DEV Community

TechSynthesis
TechSynthesis

Posted on

Automagically creating a Sitemap for React, Node.js, MongoDB website

Sitemaps play an important role in SEO optimisation. Thus having a current sitemap is important. Automating a sitemap is easy. My sitemap generator runs on my node.js web server, automatically updating my sitemap.xml in my create-react-app website, public folder when I create or delete a page from site.

Step 1. Create a new generate-sitemap.js file

Step 2-A. Import the following

import { writeFileSync } from "fs";
import mongoose from "mongoose";

//Import the model which you want to autogenerate. 
import Product from "../models/productModel.js";
Enter fullscreen mode Exit fullscreen mode

Step 2-B. Connect to your DB

const mongodbUrl = config.MONGODB_URL;
mongoose
    .connect(mongodbUrl, {
        useNewUrlParser: true,
        useUnifiedTopology: true,
        useCreateIndex: true,
    })
    .catch((error) => console.log(error.reason));
Enter fullscreen mode Exit fullscreen mode

Step 2-C. Create the sitemap generator function

export const generateSitemap = async () => {
    const products = await Product.find();
    const sitemap = ``;
}
Enter fullscreen mode Exit fullscreen mode

Step 2-D. Add the static pages

(non-changing urls i.e About, Home, Contact, Search, TOS, Privacy Policy, etc).

const generateSitemap = async () => {
    const products = await Product.find();
    const sitemap = `
    <?xml version="1.0" encoding="UTF-8"?>
    <urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
    <url>
        <loc>https://www.yourWebsiteHere.com/</loc>
    </url>
    <url>
        <loc>https://www.yourWebsiteHere.com/about</loc>
    </url>
    <url>
        <loc>https://www.yourWebsiteHere.com/contact</loc>
    </url>
    <url>
        <loc>https://www.yourWebsiteHere.com/search</loc>
    </url>
    <url>
        <loc>https://www.yourWebsiteHere.com/rules/privacy</loc>
    </url>
    <url>
        <loc>https://www.yourWebsiteHere.com/rules/terms-of-use</loc>
    </url>
    <url>
        <loc>https://www.yourWebsiteHere.com/rules/shipping-and-returns</loc>
    </url>
</urlset>
`;
}

export default generateSitemap 
Enter fullscreen mode Exit fullscreen mode

Step 2-E. Add the dynamic pages after the last </url>

It can be products, categories, articles, etc.

  ${products
            .map((page) => {
                return `
              <url>
                  <loc>${`https://www.yourWebsiteHere.com/product/${page.slug}`}</loc>
              </url>
            `;
            })
            .join("")}
Enter fullscreen mode Exit fullscreen mode

Step 2-F: Write the file to your desired location.

I'm saving it to my public folder.

writeFileSync("./frontend/public/sitemap.xml", sitemap);
console.log("genarated Sitemap successfully");
    return true;
Enter fullscreen mode Exit fullscreen mode

Step 3. Recreate the sitemap every time an new product is added.

I want the generateSitemap function to run every time I add a new product. So I'm calling it after my product is created.

import generateSitemap from "../scripts/generate-sitemap.js";

productRouter.post(
    "/",
    isAuth,
    expressAsyncHandler(async (req, res) => {
            const createdProduct = await product.save();
            //calling generateSitemap
            generateSitemap();
            res.send({
                message: "Product has been added successfully!",
                product: createdProduct,
            });

    });
Enter fullscreen mode Exit fullscreen mode

You can do something similar or create a cron job to recreate the sitemap at a set interval.

I'm using node-cron for that. I've scheduled it to run every morning at 6am and I've set the timezone closest to mine.

Install node-cron using npm install --save node-cron

import cron from "node-cron";
cron.schedule(
    "0 6 * * *",
    () => {
        generateSitemap();
    },
    {
        scheduled: true,
        timezone: "Asia/Colombo",
    },
);
Enter fullscreen mode Exit fullscreen mode

Finally once your site map is generated you can access it by typing your websitename.com/sitemap.xml or if you're on localhost then localhost:port/sitemap.xml.

Complete Code for generate-sitemap.js

import { writeFileSync } from "fs";
import mongoose from "mongoose";
import config from "../config.js";

import Product from "../models/productModel.js";

const mongodbUrl = config.MONGODB_URL;
mongoose
    .connect(mongodbUrl, {
        useNewUrlParser: true,
        useUnifiedTopology: true,
        useCreateIndex: true,
    })
    .catch((error) => console.log(error.reason));

const generateSitemap = async () => {
    const products = await Product.find();
    const sitemap = `
    <?xml version="1.0" encoding="UTF-8"?>
    <urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
    <url>
        <loc>https://www.yourWebsiteHere.com/</loc>
    </url>
    <url>
        <loc>https://www.yourWebsiteHere.com/about</loc>
    </url>
    <url>
        <loc>https://www.yourWebsiteHere.com/contact</loc>
    </url>
    <url>
        <loc>https://www.yourWebsiteHere.com/search</loc>
    </url>
    <url>
        <loc>https://www.yourWebsiteHere.com/rules/privacy</loc>
    </url>
    <url>
        <loc>https://www.yourWebsiteHere.com/rules/terms-of-use</loc>
    </url>
    <url>
        <loc>https://www.yourWebsiteHere.com/rules/shipping-and-returns</loc>
    </url>
        ${products
            .map((page) => {
                return `
              <url>
                  <loc>${`https://www.yourWebsiteHere.com/product/${page.slug}`}</loc>
              </url>
            `;
            })
            .join("")}
    </urlset>
    `;

    writeFileSync("./frontend/public/sitemap.xml", sitemap);
    console.log("genarated Sitemap successfully");
    return;
};

export default generateSitemap;
Enter fullscreen mode Exit fullscreen mode

Discussion (1)

Collapse
techsynthesis profile image
TechSynthesis Author

If you have issues with the post or if you get stuck at any point do let me know. This is my first time documenting something like this.