Next.js and Tumblr as a CMS Part 2: Data Fetching

Welp, I thought I could start a series of posts, get married shortly after, and not wind up with a huge delay between them. That was overoptimistic. But at long last, here's my second post on using Tumblr with Next.js. Check out the first one for background on why Tumblr and thoughts on organizing repos or keep reading for pointers on fetching data (with a healthy smattering of mistakes I made that you should avoid).

The Pages

Assuming you’ve already bootstrapped a Next.js app (if you haven’t, follow the instructions here), the first step is figuring out which pages you’ll need to re-create to match your existing Tumblr. For a basic blog, your pages directory should look something like this:

pages:
  ├── page:
  │   └── [page].js
  ├── post:
  │   └── [...id].js
  ├── tagged:
  │   ├── [tag]:
  │   │   └── page:
  │   │       └── [page].js
  │   └── [tag].js
  ├── 404.js
  └── index.js

If your Tumblr has custom pages, you’ll add those directly under /pages. One thing to keep in mind, the Tumblr API doesn’t support saving user questions or submissions so Next.js might not be the best solution if you need that functionality.

Connecting to Tumblr

We'll be using the tumblr.js NPM package to connect to the Tumblr API. Install it with:

yarn add tumblr.js

For authentication, you'll need to get an OAuth consumer key, consumer secret, token, and token secret. Head over to Tumblr and register a new application to get your consumer key and consumer secret. Don't stress over that form too much, you can ignore any non-required fields.

Once you have those values, enter them in the Tumblr API console and click authenticate to get your tokens. Then copy all four and save them to an .env.local file for use in your Next.js app:

TUMBLR_CONSUMER_KEY=****
TUMBLR_CONSUMER_SECRET=****
TUMBLR_TOKEN=****
TUMBLR_TOKEN_SECRET=****

If you're wondering why I recommend this method instead of using fetch with an API key, that was one of my early mistakes. Everything was fine until I finished the new site and updated my blog's visibility (hiding it outside of the Tumblr dashboard). At that point, all my requests started failing because I no longer had the proper permissions. This approach should work even if you change your settings.

Fetching Data

Now to write some actual code!

Start by creating a file called tumblr.js. I keep mine in a utils folder but you can put it wherever you'd like. This file will export a find method that uses the package we installed and the secrets we saved in the last step to create a Tumblr client and request blog posts. Here's a basic example (swap out "yourtumblr" for your Tumblr):

import tumblr from 'tumblr.js';

const CLIENT = {
  consumer_key: process.env.TUMBLR_CONSUMER_KEY,
  consumer_secret: process.env.TUMBLR_CONSUMER_SECRET,
  token: process.env.TUMBLR_TOKEN,
  token_secret: process.env.TUMBLR_TOKEN_SECRET,
  returnPromises: true
};

export async function find (limit = 10, page = 1, id, tag) {
  const client = tumblr.createClient(CLIENT);
  return await getPosts(client, limit, limit * (page - 1), id, tag);
}

function getPosts (client, limit, offset, id, tag) {
  return new Promise ((resolve) => {
    client.blogPosts('yourtumblr.tumblr.com', { limit, offset, id, tag })
      .then((response) => resolve(response))
      .catch(() => resolve({}));
  });
}

The find method accepts limit, page number, post ID, and tag params so it can be reused across all the Next.js pages listed above.

Static Props & Paths

Using the same find method means writing similar getStaticProps and getStaticPaths blocks for every page so I'll just run through index.js and post/[...id].js here. If you want more examples, check out the pages directory in my blog repo (although I do a little additional post parsing that I haven't mentioned yet).

The index.js page uses the default params and is fairly simple:

import { find } from '../utils/tumblr';

...

export async function getStaticProps () {
  const response = await find();

  return {
    props: response,
    revalidate: 3600
  };
}

Things get a little more complicated in post/[...id].js. We'll be using incremental static regeneration to avoid hitting Tumblr's 300 API calls per minute rate limit during the build process. (My second early mistake: when I first started, ISG was brand new so I tried other workarounds for limiting requests like adapting this method for caching data globally Although it technically worked, it caused a lot of extra deploys and stale data.)

In the code below, I'm statically generating the most recent 20 posts on my blog and revalidating after an hour (3600 seconds):

import { find } from '../../utils/tumblr'; 

...

export async function getStaticPaths () {
  const response = await find(20);

  return {
    paths: response.posts.map((post) => {
      const params = new URL(post.post_url).pathname.replace('/post/', '').split('/');
      return { params: { id: [ params[0] , params[1] || '' ] }};
    }),
    fallback: 'blocking'
  };
}

export async function getStaticProps ({ params }) {
  const response = await find(1, 1, params.id[0]);

  if (!(response.posts || [])[0]) {
    return { notFound: true };
  }

  return {
    props: {
      post: response.posts[0]
    },
    revalidate: 3600
  };
}

You can play around with the number of posts per page and the time between revalidations to see what works for you. If Tumblr isn't able to find any posts with the current ID, the 404 page renders instead.

That's it for now! Up next, sitemaps and RSS.