Implementing GraphQL in an existing code base

Motivation

At Kontist, we provide a mobile banking app for freelancers. As our app grew in popularity, more and more of our external partners wanted to integrate with our services. To add to our offering, we developed a browser-based web application. Soon, our customers wanted an API as well. However, our clients had different requests for an API. Each client had its own idea about what data should be returned and how. Previously, we only had a custom-tailored REST-API for our own mobile application, hardly able to fulfill all those needs.

One problem was that the API endpoints did not return the data that was required by the client. Either it was not available at all or the client was required to do a lot of follow-up requests to find a simple value (also called underfetching). On the other extreme, it returned too much data. Much of this data was never used by the client at all, having a negative impact on the performance (so called overfetching). Both underfetching and overfetching symptoms are common problems for traditional REST-APIs.

What does GraphQL

Several options exist to overcome these overfetching or underfetching. One popular solution is called GraphQL.

The main idea of GraphQL is that clients define the data structure.

Instead of enforcing a server defined data structure with multiple endpoints, each client can define what data it needs only via one endpoint.

Example REST vs. GraphQL

Let's say we want to pull up an overview of the last three account transactions. In our UI, we only need the description and the amount of the transactions.

In the REST-API we would need (at least) two requests:

1. Get information about the accounts to which the user has access, then filter the response for the primary account.
2. 2. Using the ID of the main account, pull all transaction data for this account. Then, drop all transactions except the three transactions we want to show. Drop all other data associated with those three transactions besides description and amount (e.g. we never show the booking date, the sender, IBAN, et cetera).

(Traditional approach with REST-API)

In contrast, GraphQL could fetch this transaction data like this:

(Approach with GraphQL-API)

1. Send one request with response structure defined in the body:

{
  viewer {
    mainAccount {
      transactions(first: 3) {
        edges {
          node {
            description
            amount
          }
        }
      }
    }
  }
}

In the body we use the GraphQL query language; it defines how the response should look. The available fields and their types are defined by the server up front. This allows mistakes in the client code to be detected before runtime of the application.

This approach clearly has some benefits:

• Avoid over- and underfetching
• Performance improvements because of ** less usage of network bandwidth ** no need to fetch unused data on the server ** no need to clear client data or pollute client memory
• Integrated validation and type-checking
• Documentation can be auto generated
• Easier evolution of the API (without versioning)

Implementation

API First

Introducing the GraphQL-API was more complex than just adding a small component to our code base; we started using a different development model, called "API First.” Previously, we only added new features to our mobile application then adjusted the backend for it. Now, the feature will be used by different clients at launch (web application, mobile application, our external partners and users). API First means we carefully design the model in our API, which then can be used by our own and all other clients.

Reusing code

When we started the GraphQL-API we already had a well-functioning system. It did not make sense to start from scratch; this would have cost resources and introduced new bugs. Our idea was to reuse the existing models where it was possible. We did this by introducing decorators (also called annotations) to the code of the models to expose certain properties via GraphQL. Alongside GraphQL, we introduced a new ORM version which allowed us to use decorators. Our models are now mostly clean POJOs with some decorators added.

class Account {
    @Column(DataType.INTEGER)
    id: number;

    @Field()
    @Column
    iban: string;

    // ...
}

Here we see a small excerpt of our account model class with the properties id and iban. The @Column decorator comes from our ORM and the @Field decorator is used to expose this property in GraphQL. This example shows, that we do not expose the internal database ID property via GraphQL.

Beside these decorators, we have resolver classes which find the correct entities for a given field in the GraphQL-tree.

All of this allowed us to introduce GraphQL very quickly without much overhead or boilerplate code. Instead, we could spend the time on more advanced topics like API-design, paging, sorting, and filtering. There was already a common standard for paging called connection pattern. We implemented our own solution for sorting and filtering which is now published as an open source library called type-graphql-filter.

Common problems

When dealing with GraphQL common problems are performance and security. These become a concern simply because you can do so much in just a single call. We implemented the following measures to mitigate these risks:

1. Flat structure: We do not allow recursive structures in our queries. The hierarchy for a transaction is user > account > transaction, but then this transaction entity has no property leading back to its account or user.
2. Implementation of the data loaders standard: For performance intense properties, we cache the database result during the same request.
3. Hard limit: To avoid misusage of our API, we implemented limits on request and response sizes as well as on the number of items that we return in one response.

Conclusion

The feedback from both our internal and external users of GraphQL-API was very positive. We have the ability to quickly develop new models. Currently, we are migrating all remaining REST-endpoints to GraphQL. Then we will shutdown the old legacy API.