The Code Speaks for Itself – Generating API docs for Spring Applications

Introduction

Programming is mostly about communication, and one of the most time-consuming parts of this aspect of development is the communication of how service APIs function. If this is done poorly, then the documents can get out of date, or be so vague that the developers will spend too much time answering questions about how their API works.

This post outlines a process that we in Sonalake have found to automate the creation of REST API documentation. It’s done in such a way that it won’t require too much in the way of manual effort once it’s started, because most of the documentation detail will come from work you’re already doing to test the service. We have provided a working example of this in the sonalake-autodoc-example project.

What drove the creation of this process was the aim to provide a good developer experience (DX) to our own developers, and our clients and partners, by delivering good documentation that:

1. Describes what, specifically, is in the API
2. Provides examples of how to use the API
3. Contains a changelog for how the API has evolved between versions

Tools like Swagger do a great job on automating documentation for point 1, but when it comes to points 2 and 3, these types of documentation are generally written manually (or more likely, not written at all).

By generating documentation from the source, we have found that it allows for significant portions of the API documentation to be automatically generated. By generating documented examples from unit tests, we can ensure that these examples always align with the reality of the application.

It also allows for developers to keep documentation up-to-date without having to leave the development environment, and for documents to be released and published the same way as any other development artifact.

How do we do this?

Some parts of the documentation are written manually by the developers in AsciiDoc. These parts of the documentation are not expected to change much between releases, and are limited to things like:

• Introducing what the API is
• Describing how to authenticate
• Outlining a generic set of use case steps, without any actual code samples (the code samples will be auto-generated during the build, using the data passed to unit tests).

The rest of the process will generate the following sections, also in AsciiDoc format.

• Swagger documentation concerning the paths and entities
• Code samples for the use case steps, generated from the unit tests
• Changelog history of differences between published versions of the swagger.json

Finally, the AsciiDoc files are collated and published in a single PDF file using Asciidoctor PDF.

At a high level, the main steps are as follows:

We have a developed sample project to showcase all of these steps: sonalake-autodoc-example. This is a very simple Spring Boot application with a trivial REST API with two GET methods. The rest of the project is solely dedicated to automating the documentation. Let’s walk through it.

Define Theme

The main tool for the AsciiDoctor-to-PDF generation is AsciiDoctor-PDF and it comes with a full set of theming options. The simple-theme.yml sample provides a simple, clean professional layout, that you can probably re-use by just changing the logo image.

Generate Example Code Snippets

This part of the pipeline generates snippets in AsciiDoc format from unit tests. The output contains examples of REST calls, with request bodies and responses that will always be accurate for the current version of the code base.

In the sample project this all happens in BaseWebTest. It takes advantage of spring-restdocs and acts as a base class for all other web-based unit tests.

API calls would be tested in the normal way:

mockMvc.perform(
    get("/api/endpoint-a")
    .contentType(MediaType.APPLICATION_JSON)
    .accept(MediaType.APPLICATION_JSON)
    .characterEncoding(StandardCharsets.UTF_8.name())
).andExpect(status().isOk()).andReturn();

A unit test of the form above generates the following snippets to

build/generated-snippets/${test-class-name}/${test-method-name}

• curl-request.adoc
• http-request.adoc
• http-response.adoc
• httpie-request.adoc
• request-body.adoc
• response-body.adoc

For example, a http-request for a POST might look like:

[source,http,options="nowrap"]
----
POST /api/endpoint-a HTTP/1.1
Content-Type: application/json;charset=UTF-8
Accept: application/json
Content-Length: 52
Host: autodoc.sonalake.com
{
  "fieldA" : "sample A",
  "fieldB" : "sample B"
}
----

These files can be referenced in your examples documents, with the result that examples will always be up-to-date.

Generate swagger.json

This part of the pipeline generates an up-to-date view of the REST paths and entities in AsciiDoc format. First by generating a swagger.json, and then translating this into AsciiDoc.

The sample project contains a single test, GenerateDocumentationTest.java, that starts up the application as @SpringBootTest in the test profile, and pulls down the swagger.json generated by the SwaggerConfig.java. It then runs swagger.json through the swagger2markup-gradle-plugin to convert it to AsciiDoc format.

This produces the following sets of files:

• Overview.adoc

Contains some metadata from application.yml such as title text and version information for inclusion on the main page

• security.adoc

a simple page describing how authentication, for example HTTP headers, should be configured

• paths.adoc

A list of all the REST calls and responses the application will accept and respond with

• definitions.adoc

a list of all the entities the application will accept and respond with

Make Documentation Easier to Follow

Tags are an optional, but useful, tool for collecting related endpoints together, even when they are implemented in different classes. By default, Swagger will name the resources after their controller classes, but tags allow you to give them a different name.

For example:

@Api(tags = {"Section A"})
@Description("Some operations in section A")
public class ControllerA1 {

Generate Changelog

The last part of the automated process is related to how to create a changelog. It assumes that previously released versions of the Swagger are published under Nexus. All of this configuration is contained in build.gradle.

1. Publish Swagger Spec as a Nexus Artifact using the maven-publish and maven-publish-auth plugins.
2. Generate changelog from Nexus history using the Sonalake swagger-changelog Gradle plugin. The plugin will retrieve any previously published RELEASE versions of the Swagger spec, and will produce the following:
   • A file of the form change-log-0.0.1-0.0.2-SNAPSHOT.adoc for each version
   • An index file, change-log.adoc, listing all versions

Author Hand-written Content

Writing the following documents will round out the process.

• introduction.adoc – a simple one or two paragraph description of what the applications is for
• security.adoc – a quick description of how to authenticate, and what, if any, roles exist in the application. Do note, that if you want to, you can easily write other tests that will print out a list of such roles in AsciiDoc format, and include them in this file.

A special case of a hand-written document is the examples.adoc where a high-level description of the overall flow of REST calls would be written. For example, to on-board a new user, you need to call X, then Y, and the Z. However, this document would not include any actual REST calls or parameters. Rather, it would refer to the results of the unit tests that you have written to test these endpoints.

[[examplesscheme]]
== Examples
What follows are some examples of the API usage
=== Endpoint A
A get call
include::{snippets}/endpoint-a-test/test-get-value/http-request.adoc[]
Returns this
include::{snippets}/endpoint-a-test/test-get-value/http-response.adoc[]

Since the overall flow of your application isn’t likely to change – even if the URLs and request/responses change – this document will remain relatively unchanged over time. The only thing you are likely to have to update are your unit tests, but you’d be doing that anyway. Right?

Tying it All Together

All this work is done in build.gradle – it dictates where to write the files in the build directory.

ext {
 asciiDocOutputDir = file("${buildDir}/asciidoc/generated")
 swaggerOutputDir = file("${buildDir}/swagger")
 snippetsOutputDir = file("${buildDir}/generated-snippets")
}

The following tells Gradle to pass system properties down to the test tool, so the generate documentation task can know the current document version.

test {
 systemProperties = System.properties
 systemProperty 'sg.api.version', version

 useJUnitPlatform()
}

Then use swagger2markup to convert the swagger.json into AsciiDoc format

convertSwagger2markup {
 dependsOn test
 swaggerInput "${swaggerOutputDir}/swagger.json"
 outputDir asciiDocOutputDir
 config = [
   'swagger2markup.pathsGroupedBy'                          : 'TAGS',
   'swagger2markup.extensions.springRestDocs.snippetBaseUri': snippetsOutputDir.getAbsolutePath()
 ]
}

Next, the following tells the swagger changelog plugin from where to pull version information, and to where to write the diff files.

swaggerChangeLog {
 groupId = "${rootProject.group}"
 artifactId = "${rootProject.name}-API"

 // where to find the nexus repo
 nexusHome = 'http://atlanta.sonalake.corp:8081/nexus'

 // where to store the changelog
 targetdir = "${buildDir}/asciidoc/generated/changelog"

 // if we’re building a snapshot version, then include it as the
 // end of the changelog
 snapshotVersionFile = "${buildDir}/swagger/swagger.json"
}

Finally, this is where the AsciiDoctor-PDF Gradle plugin takes all the AsciiDoc files we have created, and converts them into a pdf.

Note that the baseDirFollowsSourceDir setting, all paths are relative to the main index file. This is done because it allows for references within the AsciiDoc file structure to not have to worry about where they are on the file system.

// create a PDF from the asciidoc
asciidoctorPdf {
 dependsOn convertSwagger2markup
 dependsOn generateChangeLog

 baseDirFollowsSourceDir()

 sources {
   include 'api-guide.adoc'
 }
 attributes = [
   doctype        : 'book',
   toc            : 'left',
   toclevels      : '3',
   numbered       : '',
   sectlinks      : '',
   sectanchors    : '',
   hardbreaks     : '',
   generated      : '../../../build/asciidoc/generated',
   resources      : '../../../src/main/resources',
   snippets       : '../../../build/generated-snippets',
   changes        : '../../../build/asciidoc/generated/changelog',
   imagesdir      : 'theme',
   'pdf-stylesdir': 'theme',
   'pdf-style'    : 'simple-theme.yml',
   revnumber      : version
 ]
}

That’s it. You can take the code from the sample project into any Spring Boot project in about an hour, and produce professional, clean documents. We hope you find it as useful as we do!