When I signed up to dev.to, I skim read the terms and conditions and missed the crucial point that articles must contain the full text and not just the links. My own site is built on Hugo, which I used to create my first dev portal and the source text is in Markdown. So for most articles I've been able to copy the original across. I've lost syntax highlighting and Mermaid diagram support, but the crucial info is there. However, in the case of this article, which was originally a three part series, neither editing the Markdown nor copying and pasting from the published site worked. Therefore, this is an abridged version of the original (which you can find on my personal dev blog: Byte High, No Limit.
In this article, I'll outline how to create a fully featured dev portal for your Swagger or OpenAPI 3.0 content without spending a dime.
Part I: Backend
A dev portal is just a website that presents API (and other) docs to developers. It can be as simple as a static web page or as complex as you want to make it. In this series, we'll make use of the following back-end technologies:
- Static Site Generators
- Source control
- Containerization
- Automation
In addition, you'll inevitably spend a fair amount of time using a browser. For better or worse, Chrome is the new standard.
Static Site Generators
Hugo
Hugo is a type of web server known as a static site generator (SSG). It's lightweight and fast and can serve dynamic and static content. You can test content locally before deploying it. For OpenAPI content, we'll use the command line version of ReDoc to convert Swagger JSON files to a static HTML page that will be served by Hugo.
Install CLI tools (macOS)
- Homebrew is a package manager. From the Terminal, enter
/usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
- Git provides source control. From the Terminal, enter
git
and follow the instructions to install Xcode. - Hugo is a static site generator. From the Terminal, enter
brew install hugo
. - NPM is a package manager for JavaScript. From the Terminal, enter
brew install nodejs
. ReDoc renders an OpenAPI file into a static HTML page. From the Terminal, enternpm install -g redoc-cli@0.13.2
. You can leave off the@<version_number>
, but I've found npm doesn't always get the latest version if you do. Ensure you are using version 0.9.8 or later:redoc-cli --version\
. - (Optional) Swagger2PDF converts an OpenAPI JSON file into a static PDF document. From the Terminal, enter
npm install swagger-spec-to-pdf
.
Install CLI tools (Windows)
Requirements
- Powershell 3 (or later). Powershell 5 is already installed in Windows 10.
- .NET framework 4.5 (or later).
Instructions
- Download and install Scoop.
- Git provides source control. From PowerShell, enter
scoop install git
. - Hugo is a static site generator. From PowerShell, enter
scoop install hugo
. - NPM is a package manager for JavaScript. From PowerShell, enter
scoop install nodejs
. - ReDoc renders an OpenAPI file into a static HTML page. From PowerShell, enter
npm install -g redoc-cli0.13.2
. You can leave off the@
<version number>
, but I've found npm doesn't always get the latest version if you do. Ensure you are using version 0.9.8 or later:redoc-cli --version
. - (Optional) Swagger2PDF converts an OpenAPI JSON file into a static PDF document. From PowerShell, enter
npm install swagger-spec-to-pdf
. - cURL is a command line data transfer tool. From PowerShell, enter
scoop install curl
.
Test a local copy of the dev portal
Navigate to the folder where you have cloned the Git repository.
- From the command line, enter
hugo server
. - Copy the URL from the console output and paste it into the Chrome browser.
Add Google Analytics to a static HTML page
Paste the following text after the <head>
tag:
<!-- Global site tag (gtag.js) - Google Analytics --><script async src="https://www.googletagmanager.com/gtag/js? id=<yourID>"></script>
Source control
Git
If you are generating comments from code, you'll be working directly in the software repository. Typically, this will be a Git repository. You should familiarize yourself with the processes for creating branches, creating pull requests, resolving conflicts and merging changes. There are a number of commercial hosting options including GitHub, GitLab and Bitbucket. Here we'll look at Bitbucket.
Get the URL from Bitbucket
- In Chrome, navigate to the required repository.
- In the left icon menu, click Clone.
- Copy the URL.
Clone the repository and create a local branch
- Open PowerShell / Terminal and navigate to where your local repository folder. Example:
~/development/
. - Enter
git clone
and paste the path you copied from Chrome. This creates a local copy of the repository. - In Visual Studio Code, open the repository's folder.
- In the lower left corner of the window, click Branch.
- Enter a name in the box (
feature/apidocs-
<api class name>
) and select the branch to base it on. Example:development
You can now make your changes.
Publish changes
- In Visual Studio Code, in the left icon menu, click Source Control.
- Enter a short description of why you made the changes in the box.
- Click the Commit (tick) icon. If prompted to configure Visual Studio Code to automatically stage changes, you should do so.
- In the lower left corner of the window, click Publish Changes.
Create a pull request
- In Chrome, navigate to the Bitbucket dashboard.
- From the Repositories menu, select the repository you are working with.
- From the left icon menu, click Create pull request.
- Select your local repository from the Source list.
- Confirm the Destination repository (typically, development) and click Continue.
- Enter a Description. This should contain any additional explanation for the change.
- Attach a copy of the ReDoc HTML file.
- Select Reviewers. Typically, this is pre-populated. You can start typing a name to find a user. Ensure the product owner is included.
- Click Create.
You'll receive email notifications when the status of the pull request changes. Example: When the change is approved.
Resolve conflicts
- In your local branch:
git pull origin master
. - Merge incoming code changes while retaining doc changes.
-
git commit -am "
<your commit message>
"
git push
Merge changes
In Chrome, navigate to the pull request and click Merge.
Delete branch
- After the change is successfully merged, from the left icon menu, click Branches.
- Locate the branch you were working in and from the Actions menu, select Delete
branch.
- Click Delete to confirm your action.
Edit text in Bitbucket
While not directly related to APIs, you may on occasion have to modify text contained in a file in a Git repository stored in Bitbucket. You must have a Bitbucket account to edit the files.
- Navigate to the file in Bitbucket.
- Click Edit.
- Make your changes.
- Click Commit.
- (Optional) Enter a title in the Commit message box.
- Select the Create a pull request for this change check box.
- Click Commit.
- (Optional) Enter a Branch name. Example:
feature/uitext
. - Click Create pull request.
- Enter a Description.
- Click Create.
When your pull request has been reviewed and there has been at least one successful build, providing that there are no merge conflicts, you can merge your change.
- Navigate to the pull requests.
- Locate your pull request and click its hyperlink.
- Click Merge.
If the merge was successful, you should now delete your branch.
- Navigate to the branches.
- Click your branch's Actions button and select Delete branch, then click Delete.
Issue tracking
Track documentation tasks in Jira
- Standalone documentation tasks should have the issue type Documentation.
- Developer tasks that require documentation should have the label Documentation.
- Documentation should be part of the definition of done.
Containerization
Containers are an operating system-level virtualization technology. Sun Microsystems' Solaris Zones were an early implementation in 2004. But the technology was popularized by Docker containers, release in 2013, and Kubernetes container management, released the following year.
You don't need containerization if you're using a hosting solution such as Netlify. However, if you are hosting your own site, then you should at least consider using containers.
Install Docker
If you deploy your dev portal in a Docker container, you can try it locally before you build your automation tool chain with Docker Desktop.
After installation, you can run Docker images locally. Example: docker run -d --name rabbitmq -p 15672:15672 -p 5672:5672 bitnami/rabbitmq:latest
Create a Dockerfile for Hugo
The easiest way to deploy the Hugo static site is in a Docker image, as defined by a Dockerfile:
-
FROM
defines the base image (in the example Alpine, a lightweight Linux distribution). -
COPY
copies files and folders to the Docker image. -
ARG
specifies arguments for the Docker build command. -
RUN
executes commands. -
EXPOSE
informs a user about the ports used. -
CMD
specifies the component and its arguments to be used by the image.
Start Docker on login (macOS)
Some APIs may require middleware, for example a local RabbitMQ instance. It can be convenient to have Docker start the middleware when you log in. For example, on macOS:
- Open the Automator application.
- Select the Application type and click Choose.
- In the Actions menu, select Utilities > Run Shell Script.
- In the Run Shell Script section, select
/bin/bash
from the Shell menu. -
In the box, enter:
cd /usr/local/bin
while (! ./docker stats --no-stream ); do
sleep 10
done
./docker start rabbitmq
From the File menu, click Save.
Navigate to the Applications folder (or your user Applications folder).
Enter StartRabbit in the Save As box and click Save.
-
Open System Preferences and click Users & Groups.
- Select your user and click Login Items.
- Click Add (+).
- Navigate to the location where you saved the StartRabbit application and select it.
- Click Add.
- Close System Preferences.
The script will run the next time you log in. The path has to be changed because /usr/local/bin
is not part of the path for startup scripts. The script waits for the Docker daemon to start before starting the RabbitMQ container. It does this by querying Docker until it gets a response. A spinning cog is displayed in the right menu while the script is running.
If you are deploying more than one Docker instance, that's the perfect time to start using Kubernetes.
Automation
You should automate your deployments so that when your code changes, your API docs are updated. When software is developed using continuous integration and deployment, it helps if the docs are deployed that way too. A popular open source solutions for this is Jenkins.
Part II: Tools
When I wrote the original version of this guide, I was an API writer creating docs for an event-driven e-commerce solution. Event driven architecture (EDA) is something I'll talk more about in a future post. Development was done primarily in .NET and API docs were generated using Swashbuckle (also a good topic for a future post). This is known as the code first or bottom up approach. The more your code grows, the harder it is to maintain consistent APIs with this approach.
In my view, you should create your API specification and code against that. But for the purposes of this guide, I'll assume that you don't have any say in that, and you just have to get the API docs into the hands of developers. You may not even have an API schema, in which case you should create one in Postman, based on whatever API documentation you have available to you.
IDE
Visual Studio Code
If you're using windows and writing API docs directly in the code, you may be using Visual Studio. If you are not using Windows, then the next best thing is Visual Studio Code.
Before you set up your dev environment, ensure you have installed the CLI tools.
Set up a .NET dev environment
- Download and install the Visual Studio Code editor.
- Download and install the .NET Core SDK.
- Create a folder called
development
where you want to store your local repositories. - In BitBucket Server, go to the repository you want to work with and click the clone icon from the left toolbar.
- Copy the HTTP address.
- In the PowerShell / Terminal, navigate to the
development
folder. - Enter
git clone
and paste the HTTP address to complete the line.
Build the OpenAPI 2.0 (Swagger) JSON file
This assumes you have a working .NET dev environment and have correctly configured Swashbuckle:
- In PowerShell / Terminal, navigate to the src folder in your local repository.
- Enter
dotnet build
. - Navigate to the folder where the DLL was built and enter
dotnet run
. - In Chrome, navigate to the URL shown in the build output. Example:
http://127.0.0.1:5000
. - Click the
/swagger/v1/swagger.json
link to download the JSON file.
Plug-ins
Some recommended plug-ins for Visual Studio Code include:
• Better TOML by bungcip
• C/C++ by Microsoft
• C# by Microsoft
• Code Spell Checker by Street Side Software
• GitLens by Eric Amodio
• Markdown Preview Enhanced by Yiyi Wang
• Prettier - Code formatter by Esben Petersen
Preview API doc changes
Requirements
• You can create tasks to automate processes in Visual Studio Code.
• Each repository requires its own set of tasks.
• You must add .vscode/
to the .gitignore
file to any repository you intend to use tasks with.
• Separate scripts are required for macOS and Windows.
Convert Swagger JSON to ReDoc HTML
Enter: redoc-cli bundle <filename>.json
.
Running scripts
• After you have configured the scripts for a given repository, you can run them from Visual Studio Code.
• Press command+shift+B (macOS) or ctrl+shift+B (Windows) to run the build task.
• The ReDoc HTML file is opened in Chrome.
• If the server takes a long time to start, the ReDoc task will fail. After the server has started, run the ReDoc task from the Terminal menu.
Example Visual Studio Code tasks JSON file
This file should be placed inside the .vscode
folder in the Git repository and .vscode
added to the .gitignore
file.
tasks.json
{
"version": "2.0.0",
"tasks": [
{
"label": "build",
"command": "dotnet",
"type": "process",
"args": [
"build",
"${workspaceFolder}/src/Dev.Example.Com.Myproject/Devg.Example.Com.Myproject.csproj"
],
"problemMatcher": "$msCompile"
},
{
"label": "Swagger",
"type": "shell",
"command": ".vscode/swagger.sh",
"windows": {
"command": ".vscode\\swagger.cmd"
},
"presentation": {
"reveal": "always",
"panel": "new"
}
},
{
"label": "ReDoc",
"type": "shell",
"command": ".vscode/redoc.sh",
"windows": {
"command": ".vscode\\redoc.cmd"
},
"presentation": {
"reveal": "always",
"panel": "new"
},
"problemMatcher": []
},
{
"label": "Build API Docs",
"dependsOn": ["Swagger", "ReDoc"],
"group": {
"kind": "build",
"isDefault": true
},
}
]
}
Example scripts (macOS)
redoc.sh
cd ~/
rm redoc.jsonrm redoc-static.html
# wait for server to start
echo waiting for service to start
sleep 10
curl -o redoc.json http://localhost:62512/swagger/v1/swagger.json
redoc-cli bundle redoc.json
open -a "Google Chrome" redoc-static.html
swagger.sh
export CONSUL=[http://consul.dev.example.com](http://consul.dev.example.com "http://consul.dev.example.com")
cd src
dotnet build --source [http://dev.example.com/myproject/](http://dev.example.com/myproject/ "http://dev.example.com/myproject/")
cd Dev.Example.Com.Myproject
dotnet run --urls=http://localhost:62512
Example scripts (Windows)
redoc.cmd
cd ~/
del redoc.json
del redoc-static.html
# wait for server to start
echo waiting for service to start
sleep 10
curl -o redoc.json
swagger.cmd
set CONSUL=http://consul.dev.example.com
cd src
dotnet build --source http://dev.example.com/myproject/
cd Dev.Example.Com.Myproject
dotnet run --urls=http://localhost:62512
Part III: Writing
Most of this relates specifically to writing in-line docs directly in C# code using Swashbuckle. However, if you are using a different approach you can skip ahead to the section on Markdown.
Swashbuckle
Autogenerated Swagger / OpenAPI 3.0
Swashbuckle is a framework for ASP.NET Core that converts code comments into Swagger or OpenAPI 3.0 docs. If your developers write in C# and they're not already using a top-down API methodology, it's well worth a look.
Document API methods with multiple domain models
Some APIs use domain models to perform more than one task using a single API method.
Swagger shows the various domain models as different options in the Request Body Schema. However, it always shows the payload for the first domain model, regardless of the domain model selected.
For these API methods, the schema should be documented in a separate HTML page with a relative link, served by the static site generator.
Documentation for these API methods is declared in a private class as follows:
postPath.Post.Summary = "Short name";
postPath.Post.Description = "Long description";
The string must be contained on a single line. If you need to insert a carriage return you must use the HTML <br/>
tag.
The long description should contain the following text:
The request body schema shown applies only to the first listed command.
This should be followed by a relative link to a markdown page describing the various commands.
Create API docs in VS Code
With Swashbuckle, API documentation is created as comments in the C♯ source code (.cs
files). These comments are automatically converted to Swagger / OpenAPI 3.0 docs when the application is built. Most content can be written in markdown.
Tag API docs in the code
To make it clear in the code that the comments will be public facing, you can use XML comment format tags around the other comments:
/// <!--apiddocs-->
/// ...
/// <!--/apiddocs-->
API group descriptions
To enable a summary for a group, the Startups.cs file must include the following:
if (File.Exists(xmlPath))
{
c.IncludeXmlComments(xmlPath, true);
}
API methods
Typically, methods are associated with a particular API. However, some methods are inherited and must be edited separately. Documentation can be added using these tags:
-
summary
(plain text): A short description of the method. If no summary is provided, the API name is used. -
remarks
: A description of the method. -
param
: A short description of the input parameters. -
response
: A short description of the response code.
Example:
/// <!--apidocs-->
/// <summary>
/// Find product by SKU.
/// </summary>
/// <remarks>
/// Find a product where the \`sku\` is known.
/// </remarks>
/// <param name="sku">Stock keeping unit.</param>
/// <response code="400">Bad request.</response>
/// <!--/apidocs-->
[SwaggerOperation(Taqs = new[] { "Products" })]
[HttpGet(Name = GetProductBySkuRoute)]
...
Response parameters
Typically, responses for a given API are grouped in a single file. Documentation can be added using summary and example tags. You can find parameters in a project by searching files for [JsonProperty
. The example tags are always converted to a single string so they should only be used with the following types:
-
enum
(example:DateTimeOffset
) -
GUID
(globally unique identifier) string
Where you need to provide an example for other types, add a carriage return in the last line of the summary, followed by Example: . Don't use example tags for arrays. If you use an array example in the summary, you must escape the first square brace (\[
, ]
).
Example:
/// <!--apidocs-->
/// <summary>
/// Stock keeping unit.
/// </summary>
/// <example>
/// 6502_RICE_1KG
/// </example>
/// <!--/apidocs-->
[JsonProperty{required = Required.Always}]
public string Sku { get; set; }
Required parameters
When parameters are required, you can mark them with [Required]
, but the .cs
file will need a using System.ComponentModel.DataAnnotations;
declaration at the beginning of the file.:
If there is already a
[ JsonProperty(Required = Required.DisallowNull)]
then do not use[Required]
.
Example:
...
/// <!--/apidocs-->
[Required]
public bool AddProductToBasket { get; set; }
Commenting out comments
If you need to prevent the comments being converted to XML so that you can see what the method name is in the ReDoc output, you can use XML comment syntax:
/// <!-- <summary>
/// Stock keeping unit.
/// </summary> -->
In XML, certain characters must be escaped or the XML will fail to build from the comments (without warnings). For example, ampersands (
&
) and angle brackets, (<
,>
). If you need to use an ampersand with code font style, you must use HTML<code>
tags. If you use the markdown backtick (\
`) the escaped ampersand will not be converted to a single character.
Markdown
All tagged comments can contain plain text. Some can also contain markdown. Any that can contain markdown can also contain HTML. However, the way markdown and HTML are rendered will vary depending on the tag.
Mermaid
Liquid didn't seem to like this section. Here's my dev.to article on Mermaid: https://dev.to/aowen/creating-diagrams-with-mermaid-1lbj
Troubleshooting
Create a PDF from OpenAPI JSON
On occasion you may be asked to produce a version of your dev portal as a PDF file. Distilling an interactive website into a PDF is impossible. Most browsers will not even let you print the site in its entirety. However, you can use Swagger2PDF to generate a simple PDF from an OpenAPI JSON file:
swagger2pdf -s swagger.json
Ensure new features and changes are documented
Doc teams typically cover the work of multiple development teams, creating developer and user docs. It is not always possible to attend every scrum meeting and sprint review.
To ensure developer and user docs are delivered as close as possible to the sprint in which features are delivered, it is essential that the team receives timely notification.
There are three ways to do this in Jira:
- Create a Documentation sub task on a ticket.
- Create a Documentation task and associate it with a ticket.
- Add the Documentation label to a ticket.
You should configure a doc team kanban board to display tickets that meet any of these criteria. You can then watch a ticket to track its progress. Let your developers know how to create doc tickets:
- You can request new documentation for existing features by creating a Documentation task.
- The doc team will prioritize documentation requests to best meet customer needs.
- Requests for documentation that are made other than using Jira will be given the lowest priority.
Fix a broken API
Things to try:
-
Does the start of the .cs file have the following:
using Swashbuckle.AspNetCore.SwaggerGen;
-
Are there incorrectly declared types?
Type = "integer" / Type = "boolean"
-
If the comments are absent from the JSON is the
DocumentationFile
property set in the.csproj
file (should be set for debug and release). Example:bin\Debug\netcoreapp2.2\Your.ApiGateway.xml
Convert OpenAPI YAML to JSON
ReDoc requires OpenAPI 3.0 / Swagger 2.0 source in JSON format to create a static HTML page. Typically, auto-generated content, such as that produced by Swashbuckle, is already in JSON format. However, hand edited API schema are typically created in YAML.
- Navigate to https://editor.swagger.io/. Alternatively, download the Swagger Editor to use keep the content inside the corporate network.
- Select File > Import File.
- Fix any critical schema errors.
- Select File > Convert and save as JSON.
Convert Swagger 2.0 to OpenAPI 3.0 to resolve schema errors
ReDoc requires schema in OpenAPI 3.0 format. If you provide a Swagger 2.0 JSON file, it will attempt to do the conversion itself, but schema errors may cause the conversion to fail. Ideally you should fix the schema errors. However, if you require a quick approximation of how the API will appear you can try using the Mermade online converter.
Force iFrame link to open in parent window
ReDoc produces a static HTML page. The easiest way to add it to a Hugo site while keeping the navigation in the header is to use an iFrame. A better approach would be to ingest the page and dynamically recreate the whole thing. But that would require work by an experienced Hugo developer.
The quick and dirty approach is:
- Add
<base target="_parent">
in the<head>
section of the ReDoc HTML file. - Put the HTML pages in Hugo's
static
folder. - Create a markdown page in Hugo's
content
folder:
`
--- ---
`
Prevent API method links going to the wrong place
In the API docs, clicking a method link in the left pane in ReDoc should take the user to the appropriate method. However, when developers reuse Operation IDs, the link will instead take them to the first instance of that Operation ID. Developers must either use Operation IDs that are unique across the entire API gateway, or use no Operation IDs at all.
Remove .DS_Store files from a Git repository
- In the shell, navigate to the root of the repository.
- Enter
find . -name .DS_Store -print0 | xargs -0 git rm -f --ignore-unmatch
. - Commit changes.
Install Java 8
Oracle changed the license for JDK 8, effective 16 April 2019. You can still download it, but if in doubt, you can use penJDK 8 instead (https://www.azul.com/downloads/zulu/).
Run Windows apps on Intel Macs
Because API doc tools make heavy reliance on UNIX-land command line tools, it may be preferable to use a Mac. However, traditional technical writing tools may be Windows-only, for example MadCap Flare. The easiest way to run Windows programs on an Intel Mac is with Parallels. However, Intel Macs are going away. I'll cover the options for running Windows programs on an M1 Mac in a future post.
Run a local Rabbit MQ
Some API gateways have RabbitMQ as a build dependency. Because the remote server is not always available, you may need to run a local instance of RabbitMQ. You can use Docker to do this from the command prompt:
docker run -d --name rabbitmq -p 15672:15672 -p 5672:5672 bitnami/
rabbitmq:latest
- Navigate to http://localhost:15672/ and create a user called rabbituser with the password rabbituser as an administrator.
- Click the rabbituser user and in the Virtual Host section, then click Set permission.
- In the source code, locate the appsettings.Development.json file and change all instances of guest to rabbituser .
To restart the service:
docker start rabbitmq
To remove an old Rabbit MQ:
docker system prune -a
As a final thought, you really need a writing style guide. If you don't have one you should create one. I'll cover that topic in a future post.
Top comments (0)