DEV Community

Jayson DeLancey
Jayson DeLancey

Posted on • Updated on

Fetching Blog RSS Feeds for Building a GitHub Profile

Introduced in 2020, the GitHub user profile README allow individuals to give a long-form introduction. This multi-part tutorial explains how I setup my own profile to create dynamic content to aid discovery of my projects:

You can visit github.com/j12y to see the final result of what I came up with for my own profile page.

My GitHub Profile

Display Most Recent Blog Posts

The goal is to create a learning section that is populated by the most recent blog posts or articles from various sources. For example, if I publish a new tutorial like the one you are reading now I would like it to appear on my profile to improve discoverability of new content.

Learn from recent blog posts

Typically, I publish long-form blog content about what I'm working on in a few specific sites:
✔️ Dev.to
✔️ Medium.com
✔️ Dolby.io

Fortunately, each of these provides either an Atom or RSS feed so that we can subscribe.

Parsing an RSS Feed for Blog Posts

Really Simple Syndication (RSS) has been around for quite some time as a remnant of the popularity behind creating a semantic web by treating web sites themselves like data. Not too long ago I encountered a web developer who was surprised anybody still uses it, but I often use feeds to get new content notifications in Slack and I frequently use Feedly as my preferred way of reading the Internet. Technically, there is an alternative spec in Atom feeds as well.

Looking around a little bit I found rss-parser which fit my requirements to be lightweight, straightforward to use, and provided sample code that shows how to use it in a TypeScript project.

import Parser from 'rss-parser'

parser = new Parser()
const feed = await this.parser.parseURL(url);
for (const item of feed.items) {
  console.log(item.title);
  console.log(item.link);
}
Enter fullscreen mode Exit fullscreen mode

The attributes and version of the feed will largely determine the structure of the data and rss-parser will allow for the custom fields. The feeds I want to parse are pretty standard so didn't require much customization.

Similar to the gallery I want the blog posts to display a small image. To do this, I'll need to extract social card data from the website itself.

Unpacking Open Graph Social Cards

When you include the URL of a website in a social media application like LinkedIn, Twitter, Facebook or into a messaging app like Slack or Discord the link will expand into something more graphically interesting.

This happens because specific <meta> tags have been defined in the <head> of the web site.

<meta name="twitter:image:src" content="https://thepracticaldev.s3.amazonaws.com/i/6hqmcjaxbgbon8ydw93z.png">
<meta property="og:image" content="https://thepracticaldev.s3.amazonaws.com/i/6hqmcjaxbgbon8ydw93z.png">
Enter fullscreen mode Exit fullscreen mode

The description, title, author are all typically included in the RSS feed but the images are not. To solve for this, we can follow the link from the RSS feed and fetch the page itself.

Naively, I initially considered using something like axios to fetch the HTML. I could then use a library like jsdom or cheerio to parse it. Fortunately, I found open-graph-scraper which like rss-parser met my technical requirements. Additionally, as part of my evaluation criteria I noted that the project has a reasonable MIT license, a sizable number of weekly downloads, and a repos with the code that has a fair number of stars, watchers, and ongoing maintenance.

const ogs = require('open-graph-scraper')

// RSS fetch loop to retrieve each item from the feed

await ogs({url: item.link, ogImageFallback: true})
    .then((data: any) => {
        const { error, result, response } = data;
        image = result.ogImage.url;
        snippet = result.ogDescription;
    });
Enter fullscreen mode Exit fullscreen mode

In a few feeds, the snippet may be the first x characters from the story itself. In addition to the meta image, the description is often a succinct summary that is written for SEO purposes and would be more appropriate for the cards.

Fetch the latest Blog Post and Display

The last piece is not dissimilar to what is described in how-to use liquid template engine.

We fetch the most latest article from each source. The way the URL for feeds is constructed is a bit esoteric, but you can see the examples below for each of Medium, Dev.to, and Dolby.io.

  let feed_url = `https://${process.env.MEDIUM_ID}.medium.com/feed`;
  const medium = await feeds.getRecentArticles(feed_url);
  scope['medium_post'] = medium[0];

  feed_url = `https://dev.to/feed/${process.env.DEVTO_ID}`;
  const devto = await feeds.getRecentArticles(feed_url);
  scope['devto_post'] = devto[0];

  feed_url = `https://dolby.io/blog/author/${process.env.DOLBYIO_ID}/feed/`;
  const dolbyio = await feeds.getRecentArticles(feed_url);
  scope['dolbyio_post'] = dolbyio[0];
Enter fullscreen mode Exit fullscreen mode

And we can layout the results in the template. The shield adds a nice little decorator that can link to the full activity feed page while the item details are there to link to the content itself.


<td width="25%" valign="top" style="padding-top: 20px; padding-bottom: 20px; padding-left: 30px; padding-right: 30px;">
  <div align="center"><a href="https://dev.to/@j12y" target="_blank"><img src="https://img.shields.io/badge/dev.to-0A0A0A?style=for-the-badge&logo=devdotto&logoColor=white"/></a></div>
  <img src="{{ devto_post.image }}"/>
  <p><b><a href="{{ devto_post.url }}">{{ devto_post.title }}</a></b></p>
  <p>{{ devto_post.snippet | slice: 0,150 }}...</p>
</td>

Enter fullscreen mode Exit fullscreen mode

Learn more

I hope this overview helps fetch your own learning resources. The next article will dive into how the dynamic components are assembled.

Did this help you get your own profile started? Let me know and follow to get notified with updates when the next article comes out.

Top comments (0)