Dev.to articles migration to markdown files

#devto #javascript #tutorial #webdev

As part of my migration to TinaCMS, I also want to migrate all my dev.to articles to my new blog website. My primary goal is to consolidate all my blog posts from various platforms in my personal website.

Just create a new npm project, and add these dependencies:

"devDependencies": {
    "@types/node": "^20.8.2",
    "typescript": "^5.2.2"
  },
  "dependencies": {
    "axios": "^1.5.1",
    "ts-node": "^10.9.1"
  }

Another requirement is getting your API key from dev.to. Save it in a safe storage and we will use it in our code. You can store it in a .env variable, but we will hard-code it for simplicity.

NOTE: To follow this tutorial, try to read everything first. Each block of code in the following lines can be added in one js file.

Fetching dev.to articles, looping through each and converting to a markdown file

My starting entry point function looks like this:

async start() {
    // fetch your own dev.to published articles
    const res = await axios.get("https://dev.to/api/articles/me/published", {
      headers: { "api-key": "CSQVPaBYr6aEv8LR5os4m14K" },
    });

    const files: Article[] = res.data;

    /**
     * loop through each article which we are calling `file`
     * then convert each one to a valid markdown file
     */
    files.forEach((file) => {
      this.convert(
        // the first argument is the whole body of the markdown
        // we are concatenating the frontmatter + markdown body
        `${this.createFrontmatter(file)}${file.body_markdown}`,
        `${file.slug}.md`
      );
    });
  }

The convert function can be found below this article. It receives 2 parameters: the markdown data which is a string, and the fileName which is also a string. The markdown data has the frontmatter which is discussed right below, and also the body data which is preceded by --- characters.

Create frontmatter

Frontmatter is the metadata of a markdown, it can include any data. Also, since each value (like a title) can have a : character, we will add the value on the next line after the key. For example: ${key}: > \n ${data[key]}\n;

Here is the code:

createFrontmatter(
    data: Article,
    excluded: string[] = ["body_markdown", "user"]
  ) {
    // a frontmatter starts and ends with `---`
    // this is the start
    let frontmatter = "---\n";

    // this is the middle of the frontmatter
    // which looks like key: > \n value
    Object.keys(data).forEach((key) => {
      if (!excluded.includes(key)) {
        frontmatter += `${key}: > \n ${data[key]}\n`;
      }
    });

    // this is the end of the frontmatter
    frontmatter += `---\n`;

    return frontmatter;
  }

Dev.to provides a lot of kinds of data for us, but we will exclude body_markdown and user data.

Writing the strings to markdown file

convert(data: string, fileName: string) {
    fs.writeFile(`md/${fileName}`, data, (err) => {
      if (err) {
        console.log("Write to file has an error", err);
      } else {
        console.log("Write to file was successful");
      }
    });
  }

Also, create an Article TS type:

type Article = {
  title: string;
  description: string;
  published_at: string;
  path: string;
  url: string;
  slug: string;
  body_markdown: string;
};