DEV Community

Cover image for Rendering markdown with Markdoc in Rails
CJ Avilla for Stripe

Posted on • Updated on

Rendering markdown with Markdoc in Rails

Markdoc is Stripe’s newly open-sourced framework for authoring content. It’s the tools used to build the Stripe documentation and the framework on top of which many of the interesting and powerful features of the Stripe docs are built.

Every couple years, I can't resist yak shaving and rebuilding my personal site, cjav.dev. It’s built with 11ty. Currently, I'm just rendering long lists of links to blog posts, videos, and podcast episodes. It’s all haphazardly organized and I typically use CMD+F to find an old post that I need to reference. One benefit to the current system is how easy it is to author content and re-deploy. If I have an idea for a new blog post, I create a new markdown file and get to work. Once it’s got some words, no need to polish, I’ll commit and push which deploys the site to Netlify.

Over the past year, I’ve built some fun integrations with the Transistor.fm and YouTube APIs, and even built a little tool to visualize my progress towards my goals on Strava.

Some goals for the 2022 rebuild:

  1. Easy to author content
  2. Easy to deploy
  3. Easy to add API integrations
  4. Better surfacing of content
  5. Playground for automations (background jobs, small database)

I polled Twitter with a few options for a potential stack. My statistically insignificant poll shows bridgetown.rb followed closely by Markdoc + Next.js as the winners.

Some initial experimentation with Bridgetown.rb was very smooth. All of my existing content ported over nicely, the authoring experience was solid, and I could use Tailwind out of the box (h/t @andrewmcodes). It had support for API calls with Roda and I could see that working great to clone exactly what I have today. Bridgetown.rb is a great choice for any Rubyist building a personal site today.

As part of this exploration, I wanted to see if I could get the new Markdoc tools from Stripe to render my existing content and enable me to build custom Markdoc components that play nicely inside of a Rails app.

Markdoc ships with two built in renderers, one for React and one for HTML. The HTML example includes some Lit components 🔥. Since Rails already has lots of nice HTML-based tooling, I wanted to stick with the HTML renderer.

At a high level, I knew I needed to:

  1. Add Markdoc to the app
  2. Create an entry point for the Markdoc framework
  3. Add a build step that would recompile the Markdoc rendering logic if I added or removed components
  4. Create a way to call into the built JavaScript from Ruby
  5. Build some front end Lit components that work with the web components rendered by Markdoc

Let’s walk through the steps with a brand-new Rails application:

Create a new Rails application

rails new markdoc-rails -j esbuild
cd markdoc-rails
Enter fullscreen mode Exit fullscreen mode

We’ll use esbuild to build our client side JavaScript, but also our markdoc renderer.

Install Markdoc

This’ll add @markdoc/markdoc to the root package.json. Which works, but I’m not sure this is the best approach. Maybe it should be in an isolated package.json in the Rails.root/markdoc directory where we’ll wire up the rendering logic. Leave a comment if you have a strong opinion!

npm install @markdoc/markdoc
Enter fullscreen mode Exit fullscreen mode

We’ll create a new directory in the root of the rails app called markdoc this will contain the Markdoc rendering scripts and the schemas for our custom Markdoc components that we’ll use to juice up our content and make it more interactive, but still easy to author.

mkdir -p markdoc/{src,dist}
Enter fullscreen mode Exit fullscreen mode

A new file named markdoc/src/index.js will contain our source for our rendering JavaScript and might look something like this (to start):

const Markdoc = require('@markdoc/markdoc')

const tags = {};

function markdocRender(doc) {
  const ast = Markdoc.parse(doc);
  const content = Markdoc.transform(ast, { tags });
  return Markdoc.renderers.html(content);
}

globalThis.markdocRender = markdocRender;

// is this thing on?
console.log(markdocRender(`
# Testing 1.2.3.

- bang 💥
`))
Enter fullscreen mode Exit fullscreen mode

Notice that I’m attaching the markdocRender function to the globalThis. That’s so we expose that method from the built JavaScript and can access it from Ruby later. If you know a better way to do this, please share 🤗.

Depending on your node version, you should be able to run this now with:

node markdoc/src/index.js
Enter fullscreen mode Exit fullscreen mode

which prints the following to the console:

<article><h1>Testing 1.2.3.</h1><ul><li>bang 💥</li></ul></article>
Enter fullscreen mode Exit fullscreen mode

Add a build step

Eventually, we’ll want to call this JavaScript from a Rails controller using ExecJS or MiniRacer or some similar tool. None of the Ruby-to-JavaScript gems I found were sophisticated enough to know how to load npm modules with common.js or ES module syntax, so my solution is to just build the JavaScript with a watcher and have that run as part of bin/dev.

Let’s update package.json with two new build scripts, one for our client side JS (standard), and this new one that will build our Markdoc renderer:

  // ...
  "scripts": {
    "build": "esbuild app/javascript/*.* --bundle --sourcemap --outdir=app/assets/builds",
    "build:markdoc": "esbuild markdoc/src/index.js --bundle --sourcemap --outdir=markdoc/dist"
  }
}
Enter fullscreen mode Exit fullscreen mode

Then, we can update our Procfile.dev so that foreman can spin up this blog builder for us when we start the application:

web: bin/rails server -p 3000
js: yarn build --watch
markdoc: yarn build:markdoc --watch
Enter fullscreen mode Exit fullscreen mode

We’re ready to move in to Ruby 💎.

Create a controller and route

For my use-case, I wanted a route like /posts to render the list of markdown files and /posts/filename to render the content of the post. Ultimately, I might want to build a frame around the content of a post (think nav, header, footer etc.) so I want to have a show template where I’ll embed the content of a post written in markdown.

Generate a Posts controller

rails g controller Posts index show
Enter fullscreen mode Exit fullscreen mode

Create routes

# config/routes.rb
Rails.application.routes.draw do
  resources :posts, only: [:index, :show]
end
Enter fullscreen mode Exit fullscreen mode

You can look at the source on GitHub if you’re interested in how I build the index route, as I’m most excited to show you the show route where the Markdoc magic happens.

Rendering markdown with Markdoc from Ruby

First, we’ll need some markdown to play with. I created a few files in a new directory in the root of the app at Rails.root/posts:

/posts/2022-06-04-test.md looks like this:

---
title: "Test with links"
date: 2022-06-04
---

# Another test

\[back\](/posts)

\[out\](https://twitter.com/cjav_dev)
Enter fullscreen mode Exit fullscreen mode

/posts/2022-07-11-test-with-code.md looks like this (well sorta, dev.to also has a hard time with escaping backticks!):

---
title: Test with code
date: 2022-07-11
---

# This is a test

## This is a subtitle

- bullet

\`\`\`ruby
  class Cat
    def meow
      puts "hi"
    end
  end
\`\`\`

{% callout type="warning" %}
This is a warning callout
{% /callout %}
Enter fullscreen mode Exit fullscreen mode

When we visit http://localhost:3000/posts/2022-06-04-test we should see the rendered content of the first post.

In the show action of our PostsController, the name of the post is available in params[:id]. I thought about splitting up the path later so that it was based on date, maybe something like /posts/2022/06/04/test (which is what bridgetown does). For now, we’ll keep it simple.

def show
  post_name = params[:id]
Enter fullscreen mode Exit fullscreen mode

Next, we’ll need the raw markdown content which we’ll read in from the file on disk. Long term this directory could be configurable, just slapping on .md is probably a bit rigid, but it works fine.

def show
  post_name = params[:id]

  content = File.read(File.join(Rails.root, "posts", "#{post_name}.md"))
Enter fullscreen mode Exit fullscreen mode

Many of the posts on my personal site use code fences with the three backticks. Since this content is passed from Ruby to JavaScript as a string, we’ll escape those backticks. It took me several tries to figure out how to actually do this. In the second argument to gsub I was passing a double quoted string at first. A pattern that I’ve solidified in my head is: double quotes when doing string interpolation, single quotes for non string interpolation. But the core difference between double and single quotes comes down to the escape sequences for each! Checkout Tom de Bruijn’s article here for much more detail.

def show
  post_name = params[:id]

  content = File.read(File.join(Rails.root, "posts", "#{post_name}.md"))

  # Escape fences so that MiniRacer can evaluate:
  content = content.gsub(/`/, '\\\`')
Enter fullscreen mode Exit fullscreen mode

Now that we have the content of the post, we need to pass that down to JavaScript. I also tried a few things for this part starting with ExecJS and ultimately landing on MiniRacer, but ExecJS probably would’ve worked fine.

First add the gem:

bundle add mini_racer
Enter fullscreen mode Exit fullscreen mode

Then we’ll read in the built JavaScript and evaluate with MiniRacer.

# Read in the built Markdoc rendering script
entry_point = File.read(File.join(Rails.root, "markdoc", "dist", "index.js"))

# Evaluate the JavaScript
context = MiniRacer::Context.new
context.eval(entry_point) 
Enter fullscreen mode Exit fullscreen mode

Finally, we call our markdocRender JavaScript function passing in the raw markdown content we read in earlier and store it in an instance variable so that we can render that in our view template:

@rendered_content = context.eval("markdocRender(`#{content}`)")
Enter fullscreen mode Exit fullscreen mode

All together now:

class PostsController < ApplicationController
  def show
    post_name = params[:id]

    content = File.read(File.join(Rails.root, "posts", "#{post_name}.md"))

    # Escape fences so that MiniRacer can evaluate:
    content = content.gsub(/`/, '\\\`')

    # Run from the dist entry point
    # New markdoc components aren't added for my blog
    # too frequently, but when they are we need to rebiuld
    # with `npm run build:markdoc`
    entry_point = File.read(File.join(Rails.root, "markdoc", "dist", "index.js"))

    # Render as markdoc
    context = MiniRacer::Context.new
    context.eval(entry_point)
    @rendered_content = context.eval("markdocRender(`#{content}`)")
  end
end
Enter fullscreen mode Exit fullscreen mode

Our show template will work with rendering the html safe @rendered_content 🎉:

<%= @rendered_content.html_safe %>
Enter fullscreen mode Exit fullscreen mode

Screenshot showing a rendered post

Adding Markdoc components with Lit

One of the major benefits of Markdoc is the authoring system is very clean. You write markdown and maybe use a few components but for the most part, the authoring experience is focused on the content instead of the code around it. This is powered by Markdoc components which are implemented outside of the markdown files and used as higher level snippets while authoring.

In the Markdoc docs, you’ll find this Callout example:

{% callout title="Yellow Snow" type="warning" %}
Don't eat the yellow snow.
{% /callout %}
Enter fullscreen mode Exit fullscreen mode

Which could render with an icon and a yellow background or however we want all callouts across our blog to render.

This works by creating new tags that extend the markdown syntax.

We’ll define a new callout tag by creating a Markdoc schema:

// markdoc/src/schema/Callout.markdoc.js
module.exports = {
  render: 'Callout',
  description: 'Display the enclosed content in a callout box',
  children: ['paragraph', 'tag', 'list'],
  attributes: {
    type: {
      type: String,
      default: 'note',
      matches: ['caution', 'check', 'note', 'warning'],
      errorLevel: 'critical',
      description:
        'Controls the color and icon of the callout. Can be: "caution", "check", "note", "warning"'
    },
    title: {
      type: String,
      description: 'The title displayed at the top of the callout'
    }
  }
};
Enter fullscreen mode Exit fullscreen mode

To unlock tag extensions, we’ll need to update our rendering logic to pass in a configuration object during the transform step:

const Markdoc = require('@markdoc/markdoc')
const callout = require('./schema/Callout.markdoc')

const tags = {
  callout
};

function markdocRender(doc) {
  const ast = Markdoc.parse(doc);
  const content = Markdoc.transform(ast, { tags });
  return Markdoc.renderers.html(content);
}

globalThis.markdocRender = markdocRender;
Enter fullscreen mode Exit fullscreen mode

Now markdoc will render \{% callout %} tags as <markdoc-callout> web components.

Screenshot of the HTML for the markdoc-callout component

We’re done extending markdown by adding our new markdoc tag, now we’ll add support for this new MarkdocCallout webcomponent using Lit.

Since this is a client side feature, I’ll create a new directory in app/javascript/components and add a new file called callout.ts. We’ll also create an index.js file that simply imports all of our components one by one, e.g.: import './callout'.

Here we can build whatever logic we’d like to see for our custom callout:

import { html, css, LitElement } from 'lit';
import {customElement} from 'lit/decorators.js';

@customElement('markdoc-callout')
export class MarkdocCallout extends LitElement {
  static styles = css`
    .note {
      background-color: #8792a2;
    }
    .caution {
      background-color: #d97917;
    }
    .check {
      background-color: #000000;
    }
    .warning {
      background-color: #ffd848;
    }
  `;

  static properties = {
    type: { type: String },
    title: { type: String },
  };

  constructor() {
    super();
    this.type = 'note';
  }

  render() {
    return html`<p class="${this.type}"><strong>${this.title}</strong><slot></slot></p>`;
  }
}
Enter fullscreen mode Exit fullscreen mode

After updating our blog post to have this callout:

{% callout title="Yellow Snow" type="warning" %}
Don't eat the yellow snow.
{% /callout %}
Enter fullscreen mode Exit fullscreen mode

We now see this new component rendered to the screen (checkout the #shadow-root in the dev tools, too!):

Screenshot of callout component rendered

Conclusion

This was a fun exercise. Checkout the code and if you go down the Rails + Markdoc path, I'd love to hear about it. Next todo is to experiment with deployment. Since we’re calling down into JavaScript from Ruby, I’m not 100% sure how many deployment environments that’ll play nicely in. Drop a comment and share your favorite personal sites for inspiration, too!

Top comments (0)