declanTreanor for E.ON Developers

Posted on Mar 18, 2020 • Updated on Dec 9, 2021

How to Consolidate API-Documentation in a Microservices Environment

#java #docker #documentation #microservices

Consolidate multiple Swagger API documents in a single “source of truth” document to enable developers to utilise APIs more conveniently.

A common problem exists for developers of services, especially in a Microservices environment, and the problem can be described as follows.

The Problem

“A problem well stated is a problem half-solved” - Charles Kettering

Imagine a team of software engineers are working on a group of Microservices. The front-end developers are busy taking care of the aesthetics and usability, from the end-user’s point-of-view, and they need to know how to connect the app with the API in order to satisfy their, and thus the end user’s, requirements.

What if I have multiple microservices and I want my endpoints to be documented in one place? For example, for the convenience of front-end developers. The third-party documentation implementation in this case is Swagger UI. Most attempts to solve this problem that were found during the literature review, depended on the Swagger-UI-provided facility of the dropdown menu on the swagger-ui.html homepage. That solution meant that a consumer would need to manually access each swagger document in order to locate a suitable service.

Fig. 1.1 That label ‘Consolidated’, defaults to ‘Default’; it has been changed inside the source code. It is this dropdown that is used to control which of the APIs to view documentation for, in other solutions for this problem.

Literature Review

Perhaps “Glorified Google” would have been a more accurate title for this section, so with expectations managed, I’ll proceed.

As mentioned earlier, attempts to solve this problem, encountered in the past, made use of Swagger UI’s facility of the dropdown menu, in the upper-right of the homepage, as illustrated above, for selecting the API, for which to view the documentation.

Here is an example that sees a Eureka server being used to discover each service to be documented. Eureka is a service discovery tool; further information pertaining to Eureka servers can be found here. Although elegant, it was felt that it was overkill to have a Eureka instance, solely for this purpose, and a consumer would lack a holistic view of all the services at their disposal.

This is another example that, although is without a third-party service discovery implementation, leaves us with the necessity to manually select and investigate, each end-point individually.

In this solution, rather than employing a third-party service discovery tool, the developer simply configured the required end-points. This was an option that had occurred to me and this gave me the confidence to proceed with my intention.

A Solution

“The solution often turns out more beautiful than the puzzle.” – Richard Dawkins

This solution is a Spring-boot app that, while also using Springfox, consolidates all the configured endpoints (inside application.yml) into one single swagger document. It is relatively straightforward to configure:

endpoints: # a list of sample base urls, note, omit "/v2/api-docs"
  - name: UnusedName_key
    url: https://api_server1.eon.de
    username: joe_bloggs
    password: joe_bloggs_pa55w0rd!
  - name: UnusedName2_key
    url: https://api_server2.eon.de
    username: joe_bloggs2
    password: joe_bloggs2_pa55w0rd!

As is standard with Swagger-UI, the documentation process expects Json, describing each endpoint, to be present at the configured urls (+’/v2/api-doc’). There are two more considerations.

The following refers to, at least, Swagger version 2.x. It was necessary to hard-code Json as Strings to be available at "/swagger-resources/configuration/ui" and "/swagger-resources". Chrome Development Tools helped me, along with some prompting by a colleague, to come to this realisation. This will be outlined later, in the Configuration section.

It was also necessary to have the Json, specific to our APIs, available at ‘/v2/api-docs’, which made it necessary to provide a Controller with the above RequestMapping that returned a consolidated Json representation of all the APIs to be documented, which brings me to the second part of our implementation.

The consolidation of the swagger Json is achieved as follows: Google’s GSON is used to generate a Map “of” the individual Json representations of each API’s documentation.

The next step is to merge them into one Map, containing all the data. Maps, as you will see, lend themselves very well to merging. The developer can use any means they choose, to arrive at a consolidated Map representation. This, for example, will do the trick: After creating a variable called original, which is a Map that is equal to the first Map, in the list of Maps to be merged, that was passed into the deepMerge functionality

maps.forEach(map -> {
    updateInfo(titles, map); // this has no effect in the merging process, 
    //                          rather it does work relating to the final
    //                          title of the document
    for (Object key : map.keySet()) {
        if (map.get(key) instanceof Map && original.get(key) instanceof Map) {
            Map originalChild = (Map) original.get(key);
            Map newChild = (Map) map.get(key);
            original.put(key, deepMerge(List.of(originalChild, newChild)));
        } else if (map.get(key) instanceof List && original.get(key) instanceof List) {
            List originalChild = (List) original.get(key);
            List newChild = (List) map.get(key);
            for (Object each : newChild) {
                if (!originalChild.contains(each)) {
                    originalChild.add(each);
                }
            }
        } else {
            original.put(key, map.get(key));
        }
    }
});

return original;

GSON returns its own Map implementation to represent Json key-value pairs, this allows recursion to be used, as above, to merge all Maps into one super-map, which is then converted back, again using Google’s GSON, to Json for display as though it were a single API being rendered by Swagger UI. These will be displayed in alphabetical order, grouped by API.

Configuration

Configuration of this consolidated-swagger-springboot app, is handled inside application.yml. This configuration section refers, mainly to the configuration of Swagger-UI .

The Json returned at the RequestMapping "/swagger-resources/configuration/ui" contains, among other things, a 'filter' flag which, despite its value being boolean, defaults to true.

I have taken the opportunity to remove that property, so accepting the default, given that there may be a large number of consolidated APIs to search through. The only property required, in order that the Swagger document, renders correctly is ‘supportedSubmitMethods’, and, even that, defaults to all when it is set to an empty List. I have explicitly added all methods, for documentation purposes.


// /swagger-resources/configuration/ui

uiConfigurationMap.put("supportedSubmitMethods", Arrays.asList(
        "get",
        "put",
        "post",
        "delete",
        "options",
        "head",
        "patch",
        "trace"
));

I should highlight the aforementioned piece of configuration (or lack thereof), ‘filter’. It is the filtering ability provided here, that makes a strong case for this method of consolidation, over depending on the dropdown seen in other consolidation efforts.

As well as the configuration above, there is an opportunity to change some of the static values, appearing on the finished document, this is achieved by, in effect, overriding the Json located at, or returned by, the path "/swagger-resources", this is also hard-coded. In both cases of the aforementioned hard-coding of configuration, the following software coding pattern is used: A ‘protected static’ String, for example swaggerResource, was declared. Note, this is not declared final; this is because it will later be set to equal a jsonised Map, populated inside a static block, and later returned by a custom controller.


// /swagger-resources
 static {
    Map<String, String> swaggerResourceMap = new LinkedTreeMap<>();
    swaggerResourceMap.put("name", "Consolidated");
    swaggerResourceMap.put("url", "/v2/api-docs");
    swaggerResourceMap.put("swaggerVersion", "2.0");
    swaggerResource = "[" + new Gson().toJson(swaggerResourceMap) + "]";
}

The default document title is simply a semicolon-separated list of all the featured APIs. This list is first compiled and placed, as a list, on the consolidated Map, with the key “info”, (info.title). Then, this list is used to create the title as follows:

map.get(INFO).put(TITLE, String.join(SEMICOLON, (Iterable<? extends CharSequence>) map.get(INFO).get(TITLE)))

Additionally, this title is configurable, by including the following in application.yml.

swagger-vars:
  docTitle : 'Consolidated Doc, for example'

Finished Product

The following screenshot portrays the swagger UI consolidation of four connected APIs, with the user utilising the filter functionality to great effect.

Fig1.2 A screenshot of the homepage, with filtering enabled.

Dependencies

In order to get this to compile, java 12 will be required.

Inside pom.xml, I have declared the following dependencies of note.

Springfox
This allows interplay between Spring and Swagger.

<dependency>
    <groupId>io.springfox</groupId>
    <artifactId>springfox-swagger-ui</artifactId>
    <version>2.9.2</version>
</dependency>

Google’s GSON
This is used to unmarshall the json strings, and later re-marshall the merged Map back to a consolidated Json String.

<dependency>
    <groupId>com.google.code.gson</groupId>
    <artifactId>gson</artifactId>
    <version>1.4</version>
</dependency>

Summary of Process

Provide a mechanism for the user to configure the urls for the APIs to be consolidated.

Take each of the target APIs’ Json; next we convert each Json string to some Map implementation, then we merge those Maps. Finally, we convert that merged Map to Json and make that Json available for swaggerui to display.

Meanwhile, Springfox would have also made other Json available to Swagger-Ui, based on annotations provided by the developer. We simply create controllers that manage the compilation of that meta-data.

Docker

There is a docker Image available on Docker Hub, including a springboot app, and by creating a container from this image, you will see a working example of this app, which consolidates 4 APIs.

In order to create a container from this image you must first install docker on your local machine. Then, simply run the following command:

$> docker container run -p 80:8080 declantreanor/swaggerui:final

The app will then be viewable at the url http://localhost/swagger-ui.html

The username/password you’ll be prompted for is bob/bob.

Offline Mode

There is an offline mode; Assuming at least one online attempt has been made, thereafter the Json captured during that online attempt will be used for future attempts, unless, of course, that attempt takes place online, in which case, it will be business-as-usual.

There are clear use cases for this feature, which was achieved with simple File I/O.