How to build a Gatsby plugin to display your DEV posts

Note: Since my last Gatsby post I got a job at Gatsby on the open source team! This isn't an official post though.

DEV has a simple API that means you can use it as a basic CMS. It's also a good way to cross-post your DEV posts to your own site. There are already plugins that let you do this, but they don't support everything I need, and in any case it's a great excuse to learn how to write a Gatsby source plugin. Did you know that Gatsby is the #1 hottest tech skill to learn in 2020, according to Udemy?

First up, you should have a working Gatsby site. It's super quick to get started if you already have Node etc set up, but it's worth looking at the quick start. In this post we'll be using the default blog starter, so use the command:

gatsby new my-gatsby-blog https://github.com/gatsbyjs/gatsby-starter-blog 

If you need more help getting set up, check out the step-by-step tutorial. When you have a site that runs, come back here and you'll build a plugin.

You can do this directly in your site's gatsby-node.js file, but the tidiest way to add a data source in Gatsby is to create a custom plugin. You don't need to publish it to NPM, but you can if you want to share your creation with the world. You just need to create a folder called plugins in the root of your site. Then create a folder with a name that matches your plugin. The only required file is a package.json.

Call yours gatsby-source-dev, so create the file /plugins/gatsby-source-dev/package.json

{
    "name": "gatsby-source-dev"
}

A Gatsby plugin can do all sorts of things, but you're building a source plugin, which adds a new data source. You can use this data like any other in your site. You'll be using it to create pages.

Create the file gatsby-node.js in the same folder, which does the actual processing. This file will be loaded and run by Node when Gatsby builds your site. If you export a function called sourceNodes then it will be called at the stage when the data is being loaded and created.

This is the most basic sourceNodes function. It doesn't need to be async, but yours should be because you'll be loading data from an API. The first argument passed to the function is an object with lots of useful properties for performing the creation of nodes. The most important is the actions property, which lets you do the actual node creation.

// /plugins/gatsby-source-dev/gatsby-node.js

exports.sourceNodes = async ({ actions }) => {
// Do cool stuff
}

Now, you're not creating anything yet, but this is a valid (but useless) plugin. If you add it to your site config (we'll get to that later), then it will be called whenever you build the site. Now it's time to do something useful with it.

The first thing you're going to want to do is load the data from the DEV API. As this isn't running in a browser, you can't use fetch etc, but you can use axios, which is included with Gatsby.

// /plugins/gatsby-source-dev/gatsby-node.js
const axios = require(`axios`);

exports.sourceNodes = async ({ actions }) => {
    const result = await axios.get(`https://dev.to/api/articles/me/published`);

}

If you try loading this now, you will get a "401 unauthorized" error. To fix this you'll need to get an API key from DEV.

You'll be passing this key as a header, but first you need to look at how you configure a plugin. You don't want to hard-code the key in your plugin, so you need to pass it in to the plugin from your config.

To tell Gatsby about you plugin you need to include it in the plugins array in your gatsby-config.js, and it is also where you set the options.

module.exports = {
// rest of your config here
    plugins: [
        {
            resolve: `gatsby-source-dev`,
            options: {
                apiKey: `your-key-here`
            }
        },
    ]

}

If you're using the blog starter you'll see a couple of entries for gatsby-source-filesystem. You can remove the one with name: "blog" because you'll be getting your posts from DEV instead. You'll also need to remove gatsby-plugin-feed for now, as it'll need extra configuration to understand the posts once you're done.

Now, this will pass the key through to the plugin but it's still hard-coded in the source, which you should not do as this key will let anyone post to DEV in your name. The answer is environment variables, which can be imported from a local file or set in your build environment such as Netlify or Gatsby Cloud. The local variables are loaded by the package dotenv, which is already installed with Gatsby.

First create the .env file in the root of your project:

GATSBY_DEV_API_KEY=your-key-here

...then update your gatsby-config.js to load and use it:

// /gatsby-config.js
require("dotenv").config();

module.exports = {
// rest of your config here
    plugins: [
        {
            resolve: `gatsby-source-dev`,
            options: {
                apiKey: process.env.GATSBY_DEV_API_KEY
            }
        },
    ]

}

The plugin options are passed to all of the gatsby-node functions, so you need to update your function to use the API key:

// /plugins/gatsby-source-dev/gatsby-node.js
const axios = require(`axios`);

exports.sourceNodes = async ({ actions }, { apiKey }) => {
    const result = await axios.get(`https://dev.to/api/articles/me/published`, {
        headers: { "api-key": apiKey }
    });
}

The second argument passed to the function is the options object, and you're getting the apiKey option. You're then passing that as a header to the DEV API.

Now it's time to make some nodes. Nodes are the building blocks of the Gatsby data layer, and you can query them to create pages and display data etc.

You need to loop through the results and create a node for each. The nodes that you create will mostly be a 1:1 copy of the objects returned from the DEV API. However you do need to do a couple of small tweaks to make it work properly. The best way to grab a couple of properties and pass through the rest is to destructure the object:

// ... etc 
    result.data.forEach(post => {
        const { id, body_markdown, ...data } = post;
        // do stuff
    });
};

Next you're going to need some more of the helper functions that Gatsby provides, so add those to the arguments:

exports.sourceNodes = async (
    { actions, createNodeId, createContentDigest },
    { apiKey }
) => {
    const { createNode } = actions;

    const result = await axios.get(`https://dev.to/api/articles/me/published`, {
        headers: { "api-key": apiKey }
    });

    result.data.forEach(post => {
        const { id, body_markdown, ...data } = post;

        // do stuff
    });
};

Now to create the actual nodes. The two changes you need to make are to create an proper node id, and to tell Gatsby to convert the markdown in the post. The clever thing is that all you need to do is set the media type and content, and the built-in markdown plugin will do the rest. It will even let you use plugins to do custom things with the markdown. The default blog starter includes plugins to do syntax highlighting, copying linked files and more. Any images used in the markdown will be automatically downloaded and processed.

exports.sourceNodes = async (
    { actions, createNodeId, createContentDigest },
    { apiKey }
) => {
    const { createNode } = actions;

    const result = await axios.get(`https://dev.to/api/articles/me/published`, {
        headers: { "api-key": apiKey }
    });

    result.data.forEach(post => {
        // Destructure two fields and assign the rest to `data`
        const { id, body_markdown, ...data } = post;

        // Create the node object
        const node = {
            // Create a node id
            id: createNodeId(id),
            internal: {
                // Tell Gatsby this is a new node type, so you can query it
                type: `DevArticle`,
                // Set the markdown content
                mediaType: `text/markdown`,
                content: body_markdown
            },
            // Spread in the rest of the data
            ...data
        };
        const contentDigest = createContentDigest(node);
        node.internal.contentDigest = contentDigest;

        createNode(node);
    });
};

That's the plugin done! Run gatsby develop (or restart it if it's already running) and open GraphiQL and you'll find devArticles and allDevArticles in the explorer.

You might get markdown errors when you run the build. The parser in Gatsby is less forgiving than the DEV one, so you may need to fix some errors. e.g. I had to remove a colon from the frontmatter on one of my posts.

You can then use these to create pages. If you've started with the Gatsby blog starter, as shown in the docs, you'll have a gatsby-node.js file in the root of your project with a createPages function already in it. We just need to change it a little to create pages for your DEV posts.

First change the createPages function to match the one below. It's almost the same, but has a few changes to the data structure:

exports.createPages = async ({ graphql, actions }) => {
  const { createPage } = actions

  const result = await graphql(`
    query DevArticleQuery {
      allDevArticle(filter: { childMarkdownRemark: { id: { ne: null } } }) {
        nodes {
          slug
        }
      }
    }
  `)

  if (result.errors) {
    throw result.errors
  }

  // Create blog posts pages.
  const blogPost = path.resolve(`./src/templates/blog-post.js`)

  const posts = result.data.allDevArticle.nodes
  posts.forEach((node, index) => {
    const previous = index === posts.length - 1 ? null : posts[index + 1]
    const next = index === 0 ? null : posts[index - 1]

    createPage({
      path: `posts/${node.slug}`,
      component: blogPost,
      context: {
        slug: node.slug,
        previous,
        next,
      },
    })
  })
}

The default blog also has an onCreateNode function that adds a slug based on the source filename. You need to remove because it gets confused by our markdown which isn't created from files and has a slug defined already.

This is enough to create the pages, however if you run gatsby develop now you'll get a load of GraphQL errors, because the page queries are still expecting the old markdown node types. You'll need to change anywhere that queries the pages.

First go to /src/pages/index.js and change the page query to:

  query ArticleQuery {
    site {
      siteMetadata {
        title
      }
    }
    allDevArticle {
      nodes {
        title
        slug
        published_at(formatString: "MMMM DD, YYYY")
        description
      }
    }
  }

...Then update the component to:

export const BlogIndex = ({ data, location }) => {
  const siteTitle = data.site.siteMetadata.title
  const posts = data.allDevArticle.nodes

  return (
    <Layout location={location} title={siteTitle}>
      <SEO title="All posts" />
      <Bio />
      {posts.map((node) => {
        const title = node.title || node.slug
        return (
          <article key={node.slug}>
            <header>
              <h3
                style={{
                  marginBottom: rhythm(1 / 4),
                }}
              >
                <Link style={{ boxShadow: `none` }} to={`posts/${node.slug}`}>
                  {title}
                </Link>
              </h3>
              <small>{node.published_at}</small>
            </header>
            <section>
              <p
                dangerouslySetInnerHTML={{
                  __html: node.description,
                }}
              />
            </section>
          </article>
        )
      })}
    </Layout>
  )
}

export default BlogIndex

If you run gatsby develop now you should be able to load the site and see a list of your posts on the front page. If you don't, carefully check the queries and error messages.

Now you need to update the post template. There's also not too much to change here.

Go to src/templates/blog-post.js and update the page query to match this:

  query BlogPostBySlug($slug: String!) {
    site {
      siteMetadata {
        title
      }
    }
    devArticle(slug: { eq: $slug }) {
      id
      title
      description
      published_at(formatString: "MMMM DD, YYYY")
      childMarkdownRemark {
        html
      }
    }
  }

...and then edit the component to change the data structures:

export const BlogPostTemplate = ({ data, pageContext, location }) => {
  const post = data.devArticle
  const siteTitle = data.site.siteMetadata.title
  const { previous, next } = pageContext

  return (
    <Layout location={location} title={siteTitle}>
      <SEO title={post.title} description={post.description} />
      <article>
        <header>
          <h1
            style={{
              marginTop: rhythm(1),
              marginBottom: 0,
            }}
          >
            {post.title}
          </h1>
          <p
            style={{
              ...scale(-1 / 5),
              display: `block`,
              marginBottom: rhythm(1),
            }}
          >
            {post.published_at}
          </p>
        </header>
        <section dangerouslySetInnerHTML={{ __html: post.childMarkdownRemark.html }} />
        <hr
          style={{
            marginBottom: rhythm(1),
          }}
        />
        <footer>
          <Bio />
        </footer>
      </article>

      <nav>
        <ul
          style={{
            display: `flex`,
            flexWrap: `wrap`,
            justifyContent: `space-between`,
            listStyle: `none`,
            padding: 0,
          }}
        >
          <li>
            {previous && (
              <Link to={`posts/${previous.slug}`} rel="prev">
                ← {previous.title}
              </Link>
            )}
          </li>
          <li>
            {next && (
              <Link to={`posts/${next.slug}`} rel="next">
                {next.title} →
              </Link>
            )}
          </li>
        </ul>
      </nav>
    </Layout>
  )
}

export default BlogPostTemplate

Now open the site, click through to the link and you should see the post!

You could leave it there, but right now if you're using the starter, Kyle is getting all the credit for your posts. You can change that by using the data from your DEV profile.

Open the bio component in src/components/bio.js and edit it to get the data from one of the DEV posts:

const Bio = () => {
  const {devArticle} = useStaticQuery(graphql`
    query {
      devArticle {
        user {
          name
          profile_image_90
          twitter_username
        }
      }
    }
  `)

  const user = devArticle.user;

  return (
    <div
      style={{
        display: `flex`,
        marginBottom: rhythm(2.5),
      }}
    >
      <img
        width={45}
        height={45}
        alt={user.name}
        src={user.profile_image_90}
      />
      <p>
        Written by <strong>{user.name}</strong>
        {` `}
        <a href={`https://twitter.com/${user.twitter_username}`}>
          Follow them on Twitter
        </a>
      </p>
    </div>
  )
}

export default Bio

Now you should be able to run it and see your own profile.

When you deploy this you should be aware that it won't rebuild automatically when you add new posts to DEV. You'll need to manually trigger a rebuild when you post, or set your site to rebuild automatically at regular intervals.

You can check out the source code for this demo and view the result, built for free on Gatsby Cloud and hosted on Netlify.