DEV Community

Alyss πŸ’œ
Alyss πŸ’œ

Posted on

Slate, Swagger, and Silos - A Documentation Rebuild Story

In 2016, I migrated the API and front-end documentation of an e-commerce platform from Contentful to Slate as my first solo (and major) ruby project. At the time it was migrated, there were probably around 20 API endpoints documented and none of which had a swagger spec. Not only were there APIs, but there were TWO front-end frameworks that needed to be contained within the same site. My intent of this post is to discuss the specific problems this project posed, how they were tackled, and some constraints. If you are investigating static sites or Slate as a documentation solution, I hope you find value in my experiences.
screenshot of slate api documentation

Problems

As with any product, I had a wishlist of improvements. But it is easy for your eyes to be bigger than your stomach.

young puppy eating from an overflowing dog food bowl

  • Information architecture was atrocious. Users tended to bookmark pages more than suffer the structure.
  • API resource pages were split and not linked to each other.
  • No search, making the above problems even worse.
  • Code samples existed only in SDKs and clients hosted on github.
  • Basic getting started information was buried
  • Load time was meh.
  • Updating content required a user seat.
  • No SAML/SSO which mattered for a company that managed off- and on-boarding through LDAP.

That list doesn't seem so bad, but with the curse of knowledge, I knew there was more to each than meets the eye. Having the luxury of experiencing Twilio's API documentation, it was tough knowing the high standard it would be compared to.

  • Will the code samples have syntax highlighting?
  • Is the search site wide or only in-page content?
  • How will access be handled?
  • How do I manage the IA (information architecture)? Can it be easily updated?
  • What is the learning curve for new admins?

Solutions feat. Slate

Out of the box, Slate has only a single page. The index page is typically used by other Slate forks as a landing page (for multiple endpoints) or as a feet-first introduction to the API with a header acting as the introduction. Given the nature of API and front-end documentation, I needed to use this as a landing page.
screenshot of navigation bar and hero image

Information Architecture

Since Slate is based on middleman, the directory structure felt pretty straight forward. I could create subdirectories such as /api, create a new index.html within that, and boom: localhost:4567/api/. The main issues I ran into when working with this were three-fold:

  • The information architecture could easily become convoluted from a user or developer side. With two major API versions, I needed /api/v2/ and /api/v3/. On top of that, there are 25 resources or more (not including subresources) to cover in the documentation. I had to decide which was the lesser of two evils: a very long page (but one with search and side-nav) or many subdirectories and landing pages.
  • [Lesson learned way later] Pages with a lot of content can have some interesting issues in Slate. Specifically, using the navigation, some content wouldn't show up until you refreshed but the page had fully loaded. The way I had Slate deployed was via github pages, so the site was pre-built with static html. There was no JS or back-end nonsense that should have prevented rendering. I never figured this one out.
  • If I wanted multiple pages within the same subdirectory, it was relatively easy to name the file something like api-endpoint.html instead of index.html. The issue I had with this was relatively minor, but the final html rendering between various gems didn't like this in a few places (tables). It also hampered the search effectiveness without a federated search like Algolia.

Speed

There is no room to complain about speed with Slate. It is self-contained and only as bloated as you end up making it. At the time that I'm writing this, the dev branch for Slate has a way to pre-render the side navigation. During the migration work, it primarily used javascript. In testing the page speeds using Chrome Developer Tools and a variety of site speed tools, I noticed one page was an outlier. After wrangling further with DevTools, I was able to get a javascript profile of the page load which helped me identify the javascript functions that were taking the longest.

In my attempt to have every endpoint on the same page by version (/api/v2/), the js generating the side-nav was taking ages. It totaled about 1s+ on desktop. I presented a bit on using Nokogiri for pre-rendering the table of contents as I had worked on it asynchronous from Lord (Slate's creator). After implementing the Nokogiri pre-rendering, load times were πŸ‘Œ. By changing the ToC (table of contents) build function from client-side to sorta-server-side, the greatest damage was mitigated.

Search

As a static site, there was no full-site search but Slate did have in-page search with lunr.js. My only change was to the highlighting feature. I noticed with large pages, there was a significant feeling of lag after a single character had been typed. Increasing this to three characters was more likely to be relevant to the user without impacting their experience.

Since leaving the company where I implemented this, they will also be incorporating Algolia as a federated search across multiple sites. It is a great solution and would've worked well with Slate using DocSearch.

Swagger, meet Slate

two businessmen shake hands
During the development process, the company I was working for was in the midst of building a new major API version. This API version featured Swagger specs for each endpoint (and I was overjoyed for that). This presented an interesting project.

In the past, every API update was documented by hand. I had done this by reading PHP and others had done this by talking to the engineers or digging through Jira tickets and pull requests. Swagger offered a consistent format, an existing editor that provided documentation and examples, and had an ecosystem to support it. With primary contributions from a product manager that loves code, we ended up using Widdershins to consume the swagger file and output Slate-compatible markdown. Widdershins also helped create code samples using doT.js for templating.

If you're working with Slate or evaluating it and have questions, feel free to ping me regarding them. πŸ•Ί

Thank you to Elana and Nick for helping with final editing on this post.

Discussion (14)

Collapse
areebaasad77 profile image
areebaasad77

Hello Alyss. Thank you for this blog post, it was really helpful. I have very recently started documenting APIs and am also using slate for the document. One problem I am having is that I need the option API testing like in Swagger, but I can't find any option for that through slate. Is there a way I can get the API testing in slate or is there any other tool which offers the ease of slate with API testing?

Collapse
preciselyalyss profile image
Alyss πŸ’œ Author

So you are using swagger for documentation or looking for a solution like swagger?

Collapse
areebaasad77 profile image
areebaasad77

I'm looking for a solution like swagger but with better UI. I need an interface that I can change and edit to my own liking.

Thread Thread
preciselyalyss profile image
Alyss πŸ’œ Author

If you haven't seen it already, check out postman's solution. They have the ability to do contract testing for APIs as well.

Thread Thread
areebaasad77 profile image
areebaasad77

Thanks Alyss. We checked out postman but it is a paid solution. We cannot go for any paid solution right now. Is there any open source tool we can use?

Thread Thread
preciselyalyss profile image
Alyss πŸ’œ Author

If you need contract testing, I don't have any recommendations I could give that I've also tested.

Collapse
fgadaleta profile image
frag

is there a way to export from postman and import to slate?

Collapse
preciselyalyss profile image
Alyss πŸ’œ Author

Are you exporting the postman docs or the api request collection?

Collapse
fgadaleta profile image
frag

for now i'd be happy with documentation only

Thread Thread
preciselyalyss profile image
Alyss πŸ’œ Author

Sorry, I was trying to clarify what file type you'd be working with. Is there a way to export existing markdown files or swagger/OAS from postman and is that what you'll be using?

Thread Thread
fgadaleta profile image
frag

Ok, my attempt is to use Postman for debugging and writing the doc. But Slate to publish. So I'd need postman to export a json that I can somehow convert for Slate

Thread Thread
preciselyalyss profile image
Alyss πŸ’œ Author

Depending on the file output, you might have an easier time with a project that could just consume open api spec out of the box. Box custom built their dev docs and pull from their open api spec repository. You can see some code here but I don't believe an arch diagram has been published github.com/box/developer.box.com

Collapse
pandit_pratibha profile image
Pquery

Hi we are evaluating Slate for API documentation. We also have swagger yaml spec. I have seen some solutions for swagger yaml to slate markdown conversion, but that seems like having all documentarion in yaml and use slate for rendering. What I want ideally is have swagger yaml as baseline API doc that comes with dev release and be able to build more detailed documentation on top of it directly in slate. Key here is to be able to preserve both slate doc & baseline doc from swagger every release without having to redo slate documentation. Have you come across this workflow or have any tips? Thanks

Collapse
preciselyalyss profile image
Alyss πŸ’œ Author • Edited

There's only one or two options that come to mind that might work, but I haven't personally tried to tackle this problem.

If you look at how Slate handles the TOC, there's a file in the ~/lib subdirectory that does a pre-rendering of the content. So the markdown has already been covered to html and this modifies it. You might be able to get away with something similar if the swagger converted file and the baseline doc had something you could consistently match up against, like a header. The main issue with that is you may have a superfluous site path unless you buried one of the markdown files outside of the normal directory.

Alternatively, you could modify deploy.sh to use regex to merge the two files so they maintain separation in your code base until they are ready to go live. That way you have a single URL path and the documentation would use the normal middleman rendering path. Any changes to ~/deploy.sh would need to come before the run_build() function.

All of that is speculation on what might work. I haven't touched Slate in sometime, but hopefully it can get you in the right direction.