Programmatic SEO with Next.js

Looking for Programmatic SEO consulting? (strategy & implementation) Get in touch here: agam [at] unzip [dot] co

If you are interested in developer trends you should check out my new newsletter at: unzip.dev

This article will cover one of the hottest terms today in the marketing world - Programmatic SEO, and show real-life examples using Next.js.

What is Programmatic SEO?

In short, we are leveraging code to generate organic landing pages at scale, specifically for SEO.
Remember how when you search Google for the best romantic hotels in amsterdam you see booking.com up there in the top 3 search results?

The crazy fact is that they are up there for almost all locations. Try changing amsterdam to goa and most probably you will still see booking.com up there.

They achieve this by utilizing programmatic SEO - they generate 1000s of landing pages for those long-tail keywords, and thus getting tons of organic traffic for all those location keyword permutations.

For a deeper intro on the subject, how it works and why it matters, you can check out: 0x003-programmatic-seo (via unzip.dev the developer trends newsletter).

Why Next.js?

In a real-world scenario, we would generate hundreds if not thousands of landing pages, which brings up some interesting technical constraints we need to address.

Next.js can solve most of our problems by giving us optimized static landing pages, where optimized refers to the SEO “quality” of the page - read more about on-page SEO here:

✅ Pages will be loaded quickly because they are static and on a CDN. Page speed plays a crucial role in SEO rankings.

✅ Having the pages rendered on the server makes them better for crawlers and therefore our SEO.

✅ Having the ability to re-generate the static landing pages lazily when the content is stale via ISR. This way we make sure our landing pages are still relevant with minimal work on our end.

✅ As an added bonus, Next.js plays along well with React (a big plus, ecosystem-wise), is very well maintained (docs, codebase, and examples) and has a great community.

Having all those features baked into our framework makes Next.js a prime candidate for programmatic SEO projects.

Show me the final result!

The full source code of this tutorial is here: next-programmatic-seo-tutorial - (NEW REPO: pseo-next - Next 13 compatible)

We will build a mock eCommerce website with landing pages for each category of products. We will populate them with the highest-ranked products per category.

This is a dummy example, in real life you will need to do real SEO research, have more long-tail keywords and use informative data, but the basic tech details are laid out here.

When the user searches for “10 Best [category] products” they will get to our long-tail, programmatically generated landing pages.

The end result will look like this:

Let’s start!

Mocking our data

In a real-world scenario, this data will be fetched from your real DB, API, or via scraping useful information. We are mocking data only for the purposes of this tutorial.

I will use a nice package called keikaavousi/fake-store-api which sets up a fake eCommerce back-end with MongoDB, that we can use to mock our products.

The altered code for the store is here. (I took the liberty to change the code a bit to add ranking and a dummy data script).

To start it locally check the README.md I created. It will start our server and MongoDB in a docker container. It will look like this:

Now we have a working mock server to fetch data from.

Let’s design our landing pages

We need to come up with a nice way to show all the data we have, in a way that adds value to the end-user.

After doing some keyword research (see the guide here), I decided that our Programmatic SEO landing pages will be of the following format:

“10 Best [category] products” (in the real world, this will be hard to rank for, probably not long-tail enough)

Let’s inspect what a single product looks like when we fetch it from the database:

{
    "id": 1,
    "title": "Sleek Cotton Chair",
    "price": 516,
    "description": "The Nagasaki Lander is the trademarked name of several series of Nagasaki sport bikes, that started with the 1984 ABC800J",
    "image": "<http://placeimg.com/640/480/cats>",
    "category": "Books",
    "rating": 82,
    "__v": 0
}

For each product we should probably display the: title, description, rating and price. In our Next.js category page we can display them like so:

<div className="grid">
  {products.map(product => {
    return (<a href="#" key={i++} className="card">
      <h3>{product.title}</h3>
      <p>{product.description}</p>
      <p><i>Price: ${product.price}, with rating of: {product.rating}/100</i></p>
    </a>)
  })}
</div>

Generating the landing pages

First, we need to tell Next.js which URLs we want to build beforehand.

In our Next.js project, let’s create a directory with a page like so: pages/best/[category].js. Now let’s add code to fetch the data from our database.

To do so we will use getStaticPaths - it tells Next.js what pages it will need to generate. In our case, we need one per category (we have 22 categories we want to pre-generate).

Without getStaticPaths Next.js will not know what pages to build as we are using a dynamic page [category].js. In addition, we are using fallback: ‘blocking’ so Next.js will try to fetch information for pages it didn’t build already (if we have new information) instead of returning a 404 not found response. It will block until it compiles a static response, so search engine bots will get data and not have to render client side code.

export async function getStaticPaths() {

  // We need to fetch all of the categories from our DB
  const res = await fetch('<http://localhost:6400/products/categories>')
  const categories = await res.json()

  // We need to adhere to the Next.js getStaticPaths structure
  // <https://nextjs.org/docs/basic-features/data-fetching/get-static-paths>
  let paths = categories.map((x)=>{return{'params': {'category': x}}})

  // For blocking see: <https://nextjs.org/docs/api-reference/data-fetching/get-static-paths#fallback-blocking>
  return {
    paths,
    fallback: 'blocking'
  };
}

As you can see here, when we create a production build (npm run build), before even entering the site, Next.js has generated all of our pages beforehand (great for CDN'ing our files).

Automotive.html
Automotive.json
Baby.html
Baby.json
Beauty.html
Beauty.json
Books.html
Books.json
Clothing.html
Clothing.json
Computers.html
Computers.json
Electronics.html
Electronics.json

Now that we have each page per category, we need to populate the pages with category-specific data on the top 5 products per category. To do so we will use getStaticProps) which will be called during build time and will fetch data from our database.

Note that params here contains the category we previously generated with getStaticPaths.

export async function getStaticProps({params}) {

  // Let's fetch the latest top ranking items in a category from our DB
  const res = await fetch(`http://localhost:6400/products/category/${params.category}`)
  const products = await res.json()

  // Let's pick the 5 best ranked ones
  const topProducts = products.sort((a,b) => b.rating - a.rating).slice(0, 5);

  // Every time we statically generate this page we will have the time-stamped.
  const stats = new Date().toISOString()

  return { 
    props: { stats: stats, topProducts }
  };
}

Note: we also add the new Date(), so we will have an indication of when we generated the page.

Optimizing our landing pages during runtime

Using ISR, which stands for Incremental Static Regeneration, we can rebuild stale landing pages (landing pages where the data is no more relevant). In our case, these are pages where the ranking changed.

If the ranking of a product is lower than before, we don’t want to showcase it on our landing page → making our landing page more valuable as it is more up to date.

To do so we only need to add one(!) thing, the revalidate property to our getStaticProps.

return { 
    props: { stats: stats, topProducts },
    // Next.js will attempt to re-generate the page:
    // - When a request comes in
    // - At most once every 1 hour
    revalidate: 3600, // 1 hour
  };

Revalidate is really neat. Before, if we wanted to regenerate our pages content, we would have to initiate a full npm run build command to re-generate all of our categories - which could be really time-consuming if we have 1000s of pages. Each visitor request to a revalidate-enabled page will check if enough time has passed. If, in our case, an hour has passed it will fetch the data again and build that specific page. With only the new information - an awesome optimization instead of generating all the pages at once.

Note that the first user after that 1 hour will still have stale data, but the next visitors will have the new information, but this isn’t a real issue in our use case.

Adding SEO tags

We will have to populate SEO-related meta tags with the relevant information for each page. To do so, we will use garmeeh/next-seo.

<NextSeo
  title={`10 Best ${category} Products`}
  description={`10 Best ${category} Products for bestecommerce.com updated daily.`}
/>

In a real-world scenario, we will research the heck out of the SEO metadata we generate, but for this example, I just wanted to show you how to add it to the page. I will recommend some starting points:

• Ahrefs blog
• Backlinko blog

Recap

Now we have a website with 22 landing pages, optimized for changes in our data.

The next step would be to fine-tune our SEO scores for those pages, which can be done via Lighthouse.

The source code for this tutorial can be found here: agamm/next-programmatic-seo-tutorial - New Repo: pseo-next

Future

Keep in mind that this just scratches the tip of the iceberg. I will create a Next.js programmatic SEO starter template soon. Please comment if you would like something like that and I'll send it via unzip.dev 🚀

If you found this tutorial helpful or interesting I’m sure you will enjoy my Developer-trends newsletter (every 2-3 weeks or so) called unzip.dev.

Other than that if you like this type of content, please follow me on Twitter.