Forem has set a milestone to update our (v1) API documentation and we need YOUR help!

There are several endpoints that we would like to document in order to complete our v0 -> v1 upgrade. v0 will eventually be deprecated and removed (there aren't any breaking changes so existing endpoints will continue to work the same as before). If you’re looking to contribute to open source, these are awesome first issues to work on and this post will help guide you through them.

In this post, I’ll outline the details about our v1 API, we’ll discuss its documentation and then I’ll walk you through an example of an API endpoint that I had recently documented.

About the v1 API

The API is a REST (REpresentational State Transfer) API, which means that it follows a set of guiding principles that you can read more about here.

There are currently many resources that can be accessed via the API, however, it does not contain all of the resources that we have available on DEV. We are continuously adding more endpoints to the API.

It is important to note that all operations are scoped to a user at the moment, and one cannot interact with the API as an organization.

Headers

The API consists of both authenticated and non-authenticated endpoints.

Most read endpoints do not require authentication and can be accessed without API keys. However, we require authentication for most endpoints that can create, update or delete resources, and those that contain more private information.

CORS (Cross Origin Resource Sharing) is disabled on authenticated endpoints, however endpoints that do not require authentication have an open CORS Policy. With an open CORS policy you are able to access some endpoints from within a browser script without needing to connect via a server or backend.

Authentication is granted via a valid API Key. The API key can be generated by logging into your account at dev.to and clicking on Generate API Key on the Settings (Extensions) page.

Once you’ve generated your API key, you are ready to start interacting with the API. The API Key will be set as a header, namely api-key, on a request.

We require another header for accessing the v1 API - the Accept header. The Accept header needs to be set to application/vnd.forem.api-v1+json, where v1 is the version of the API. If you do not pass along an Accept header you will automatically be routed to v0 of the API. API (v0) will be deprecated soon and we encourage you to rather use v1.

Accessing the API

As mentioned above, there are some API endpoints that are authenticated and others that do not require authentication.

When interacting with an endpoint that does not require authentication, you can pass through a single header (the Accept header) that will set the version of the API that you are interacting with.

curl -X GET https://dev.to/api/articles\?page\=1 --header "accept: application/vnd.forem.api-v1+json"

An endpoint that requires authentication would need both the Accept and the api-key header to be set when making the request:

curl -X PUT http: //localhost :3000/api/users/1/unpublish
--header "api-key: <your-api-key>"
--header "accept: application/vnd.forem.api-v1+json" -V

You need to have the correct roles and permissions set on your user to be able to query certain data from the API. For example, only admins can read, create, update and delete Display Ad resources.

About our documentation

Our v1 API endpoints are documented according to the OpenAPI Specification.

The OpenAPI Specification

The OpenAPI Specification (previously known as a Swagger Specification) is a standard for defining RESTful interfaces. As per the definition on their website, it is a document (or set of documents) that defines or describes an API.

An OpenAPI definition uses and conforms to the OpenAPI Specification. The OpenAPI definition can be created within your codebase.

You can view the Open API Specification here. It describes API Versions, Formats, Document Structure, Data Types, Schemas and much more.

When an API adheres to the Open API specification, it allows opportunities to use document generation tools to display the API, code generation tools to generate servers and clients in various programming languages, access to testing tools etc. Some of these tools include Swagger UI, Redoc, DapperDox and RapidDoc.

Forem, which is a Ruby on Rails app, integrates the Open API Specification via a gem - the rswag gem. The rswag Ruby gem allows us to create a DSL for describing and testing our API operations. It also extends rspec-rails "request specs", hence, allowing our documentation to be a part of our test suite which allows us to make requests with test parameters and seed data that invoke different response codes. As a result, we are able to test what the requests and responses look like, however we do not test the business logic that drives the endpoint - that is tested elsewhere in the code.

Once we write the test, we are able to generate a JSON file that conforms to the Open API Specification thus allowing us to eventually use the document generation tools to format and beautify our documentation.

The OpenAPI definition in the form of api_v1.json is generated and used as input to Docusaurus to create our documentation. You can view our v1 documentation here.

Now that we’ve discussed how Open API and rswag fit together to create the API documentation let’s work through an example of adding documentation to an endpoint together.

Documenting a v1 endpoint

We’ll be working on this github issue together. The issue outlines a task to use rswag to document the /api/followers/users endpoint in the v1 API.

For reference, our v1 documentation lives here.

There is also a pull request for the code written in the example below which you can reference.

Skeleton

Let’s start by creating a file for the endpoint that we want to test and document.We can go ahead and create spec/requests/api/v1/docs/followers_spec.rb

There are some building blocks for each test - a skeleton that defines some standards, implementation details like generating the header values, seed data etc.

Below is a code snippet of the skeleton for this test:

require "rails_helper"
require "swagger_helper"

# rubocop:disable RSpec/EmptyExampleGroup
# rubocop:disable RSpec/VariableName

RSpec.describe "Api::V1::Docs::Followers" do
 let(:Accept) { "application/vnd.forem.api-v1+json" }
 let(:api_secret) { create(:api_secret) }
 let(:user) { api_secret.user }
 let(:follower1) { create(:user) }
 let(:follower2) { create(:user) }

 before do
   follower1.follow(user)
   follower2.follow(user)
   user.reload
 end

 describe "GET /followers/users" do
   path "/api/followers/users" do
     get "Followers" do
     end
   end
 end
end

# rubocop:enable RSpec/EmptyExampleGroup
# rubocop:enable RSpec/VariableName

We start off by importing the necessary libraries - in this case the rails_helper and the swagger_helper that will allow us to use the DSL to build out our definitions.

If you’re use RSpec before then the describe block will be familiar to you, it will create an example group.

let(:Accept) { "application/vnd.forem.api-v1+json" }
let(:api_secret) { create(:api_secret) }

Above, we define (but not yet set) the header values. The accept header will allow us to access the v1 API and since the /api/followers/users endpoint requires authentication we generate an API secret that we will use later on.

let(:user) { api_secret.user }
let(:follower1) { create(:user) }
let(:follower2) { create(:user) }

 before do
   follower1.follow(user)
   follower2.follow(user)
   user.reload
 end

Above, we use RSpec to setup our data so that we can have example responses in our API documentation. We create a user and two follower users. With these models in the DB rswag will run the tests and display the example tags created by FactoryBot in the json file. In the before block, we then setup the two follows to follow the user.

describe "GET /followers/users" do
   path "/api/followers/users" do
     get "Followers" do
     end
   end
end

In a nested describe block we start by specifying the path for the endpoint that we’re testing which is /api/followers/users. You can read more about path operations in the rswag documentation.

In some circumstances, you may have an identifier or parameter in the path. These are surrounded by curly braces. For example: /api/articles/{id}.

Operation Ids

get "Followers" do
end

The above is considered our operation block. In this instance we define that we are implementing a GET.

The Operation Object has several fields that can be set to help define this endpoint. You can read more about operation objects here.

These are some of the fields that we want to define for api/followers/users:

get "Followers" do
  tags "followers"
  description(<<-DESCRIBE.strip)
  This endpoint allows the client to retrieve a list of the followers they have.
  "Followers" are users that are following other users on the website.
  It supports pagination, each page will contain 80 followers by default.
  DESCRIBE
  operationId "getFollowers"
  produces "application/json"
end

Below, I’ve taken some of the definitions from the specification and applied it to the code sample to help explain what each field does.

tags: A list of tags for API documentation control. They are used for logical grouping of operations by resources or any other qualifier. In this case, we want this endpoint to be grouped on its own and so we provide it with a new tag. In other circumstances, you may want to tag your crud operations for a single resource all with the same tag so that they can logically group together.

description: Provides a verbose explanation of the operation behavior. CommonMark syntax may be used for rich text representation. We try to describe what the endpoint does in a single sentence. Thereafter, we can provide additional context that we think will be useful to the user of the API at a glance.

operationId: This is a unique string used to identify the operation. The id must be unique among all operations described in the API. The operationId value is case-sensitive. Tools and libraries may use the operationId to uniquely identify an operation, therefore, it is recommended to follow common programming naming conventions. We try to structure the work with the CRUD operation as the prefix and the resource as the suffix.

produces: This field populates the response content type with the produces property of the OpenAPI definition. Our response type is usually JSON.

Another noteworthy field that we add to other endpoints but is not relevant to this particular endpoint is:

security: This is a declaration of which security mechanisms can be used across the API. Individual operations can override this definition.

We define a security scheme globally in swagger_helper.rb levelsecurity: [{ "api-key": [] }],. The security scheme that we utilize applies authentication via an api-key header.

However, remember earlier I mentioned that not all endpoints need authentication. Hence, for those that do not need authentication, we can override the "security" attribute at the operation level. To do this, we provide an empty security requirement ({}) to the array.

If you look at /api/articles in the article_spec you will see security [] which indicates that this endpoint does not need authentication.

However, in the case of the endpoint that we’re documenting /api/followers/users we do not need to provide a security field as we’ll use the one that is defined globally with the API key for authentication via an api key.

Now that we’ve set these operationIds, let’s have a look at how they would get generated in the JSON file.

The operationIds set the scene for how the API operates. Next, we’ll want to define the parameters that the API endpoint allows.

The following two sections, Parameters and Response Blocks, rely on schemas, so before we get into the details of these sections let’s first discuss what a schema is.

The OpenAPI Specification allows you to describe something called a "schema" which in its simplest form refers to some JSON structure. These can be defined either inline with your operation descriptions OR as referenced globals.

You can use a referenced global when you have a repeating schema in multiple operation specs. For example, an article resource may be returned in a create, read or update, hence instead of repeating this JSON across all these operations you could add it as a referenced global in the swagger_heper.rb and then use that $ref.

The global definitions section lets you define common data structures used in your API. They can be referenced via $ref: "a schema object" – both for request body and response body.

Another instance where you may want to use a referenced global is when multiple endpoints accept the same parameter schema and you do not want to repeat this JSON for multiple endpoints.

Parameters

If you look at the code for the api/followers/users endpoint, you’ll notice that it takes three optional parameters; page, per_page and sort.

You’ll notice that we use page and per_page across multiple endpoints for our pagination strategy hence that reusability makes it the ideal candidate for a referenced global.

Since it’s been used before, you’ll find it defined globally in the swagger_helper.rb. Hence, all we need to do is reference it in our spec.

parameter "$ref": "#/components/parameters/pageParam"
parameter "$ref": "#/components/parameters/perPageParam30to1000"

However, our next parameter - sort, has not been defined before and does not seem to be re-used in the same manner across any existing endpoints. Thus we can define it inline.

    parameter name: :sort, in: :query, required: false,
              description: "Default is 'created_at'. Specifies the sort order for the created_at param of the follow
                               relationship. To sort by newest followers first (descending order) specify
                               ?sort=-created_at.",
              schema: { type: :string },
              example: "created_at"

You can read more about describing query parameters in the OpenAPI Guide here

This is what the final set of query parameters look like:

get "Followers" do
  ......
  parameter "$ref": "#/components/parameters/pageParam"
  parameter "$ref": "#/components/parameters/perPageParam30to1000"
  parameter name: :sort, in: :query, required: false,
            description: "Default is 'created_at'. Specifies the sort order for the created_at param of the follow
                               relationship. To sort by newest followers first (descending order) specify
                               ?sort=-created_at.",
            schema: { type: :string },
            example: "created_at"
end

And this is what the corresponding generated JSON would look like:

Response Blocks

Once we’ve defined the operation Ids and Parameters, we can define what the response query looks like. We can create multiple response blocks in order to test the various responses a user of the API may receive. This includes testing when the API endpoint provides a response, when there is no content, when the user is not authorized etc.

In this case, we’ll test what a successful response looks like and what an unauthorized response looks like when we do not provide the correct api-key.

A successful response with status code 200

response "200", "A List of followers" do
  let(:"api-key") { api_secret.secret }
  schema type: :array,
         items: {
           description: "A user (follower)",
           type: "object",
           properties: {
             type_of: { description: "user_follower by default", type: :string },
             id: { type: :integer, format: :int32 },
             user_id: { description: "The follower's user id", type: :integer, format: :int32 },
             name: { description: "The follower's name", type: :string },
             path: { description: "A path to the follower's profile", type: :string },
             profile_image: { description: "Profile image (640x640)", type: :string }
             }
   add_examples


   run_test!
end

The HTTP 200 OK success status response code indicates that the request has succeeded.

In order for the request to succeed, we first need to provide the necessary authentication. When this example runs, it will need the api-key for authentication, hence we set it in our RSpec test.

In this case, I’ve decided to define the schema object inline because it is a uniquely structured response that is not being shared with other endpoints. However, if more than one endpoint had the same schema it would have been beneficial to define it globally in the swagger_helper and then provide a reference to it in the various spec files.

  schema type: :array,
         items: {
           description: "A user (follower)",
           type: "object",
           properties: {
             type_of: { description: "user_follower by default", type: :string },
             id: { type: :integer, format: :int32 },
             user_id: { description: "The follower's user id", type: :integer, format: :int32 },
             name: { description: "The follower's name", type: :string },
             path: { description: "A path to the follower's profile", type: :string },
             profile_image: { description: "Profile image (640x640)", type: :string }
          }
        }

The schema that we have defined above is an array of objects. The top level type is defined by an array type and each item is an object. Thereafter, we further define the properties that can be expected in each object. We advise that a description for a property is added where necessary.

If the need to re-use the schema object for multiple endpoints arose, we could have defined it as a Follower in the swagger_helper.spec and then referenced it in our spec like below:

schema type: :array,
        items: { "$ref": "#/components/schemas/Follower" }

The add_examples method can be found in the swagger_helper and it is responsible for creating the Response Object.

It creates a map containing descriptions of potential response payloads.
The key is a media type or media type range like application/json and the value describes it.

Finally, the run_test! method is called within each response block. This tells rswag to create and execute a corresponding example. It builds and submits a request based on parameter descriptions and corresponding values that have been provided using the rspec "let" syntax. In order for our examples to add value, we want to give it a good set of seed data.

If you want to do additional validation on the response, you can pass a block to the run_test! method.

You can read more about how to use run_test! from the rswag documentation.

An unauthorized response with status code 401

response "401", "unauthorized" do
  let(:"api-key") { nil }
  add_examples

  run_test!
end

The HyperText Transfer Protocol (HTTP) 401 Unauthorized response status code indicates that the client request has not been completed because it lacks valid authentication credentials for the requested resource. Hence, in this case if we provide an invalid api key, we should expect a 401.

To test this case, we simply provide an invalid API key which will not allow us to authenticate to the API.

The generated JSON for these two responses look as follows:

Generating the JSON

I’ve been referencing the JSON files above, but you must be wondering how do you access that JSON. Once you have written your spec, you can generate the JSON file for the API by running

SWAGGER_DRY_RUN=0 RAILS_ENV=test rails rswag PATTERN="spec/requests/api/v1/**/*_spec.rb"

Once you do this, you will see a newly generate file at https://github.com/forem/forem/blob/main/swagger/v1/api_v1.json.

Take the time to evaluate the generated content in this file, especially for the new spec. In order to view it you may paste the JSON into https://editor.swagger.io/. When you do this, it will display the data as documentation and also let you know if there are any errors.

If you have Visual Studio Code, we suggest you install the following extensions that enable validation and navigation within the spec file:

• OpenAPI (Swagger) editor
• openapi-designer live preview

And that, my friends, is how we document API v1 endpoints at Forem.

You can find the code for this example here.

If you have any questions or feedback, please drop them in the comments below. If you’d like to contribute to our documentation please have a look through the ones that aren’t assigned in this milestone and raise your hand on the issue. We look forward to your contributions.