why IDL matters in API development

We are in the era of API, and our services are exposed in APIs. When there is a new business requirement coming, a context storming happens and the feature will be degined in the principle of DDD and implemented in the form of APIs. A backend developer starts his work by designing an API contract and share it with his/her frontend peers. Frontend developers or from other team design their functions by the API contract. That is to say, we must work on a solid foundation of API contract so that everyone who are in the scope can work independently and start their work in parallel.

This document talks about the the problem of API contact sharing that we face everyday, and why it is neccessary to reconsider how to improve our productivity by changing our way of API design, sharing and development.

RPC v.s. Restful

The Representational State Transfer (REST) style is an abstraction of the architectural elements within a distributed hypermedia system. - Roy Thomas Fielding.

A RESTful API focuses on the resources that an API client can require from the API service. It is resouce based and usually performs CRUD operations on resource endpoints. A typical RESTful API usually looks like:

POST /users/501/messages HTTP/1.1
Host: api.example.com
Content-Type: application/json

{"message": "Hello!"}

In Contrast, RPC API is action based, which only have get and post functions.

POST /SendUserMessage HTTP/1.1
Host: api.example.com
Content-Type: application/json

{"userId": 501, "message": "Hello!"}

In Contrast, RPC API is action based, which only have *get* and *post* functions.

Use RESTful to replace RPC?

Let`s examine the following example to see if it a good solution.

RESTful Approach

Imagine we are developing a library system that needs to handle the status of titles. When the library admin needs to update the status of tiltles, you may provide APIs like below.

• POST /titles/123/in
• POST /titles/123/out
• POST /titles/123/suspended

`shell
PATCH /titles/123 HTTP/1.1
Host: api.example.com
Content-Type: application/json

{"status": "Out"}

`

There is a problem that you want to update mutilple titles in batch and want to add background logic when you update a specific status. For example, when you want to set a title to state of "In" and notify all the readers that are in the wating list.

RPC Approach

You can design your API with action based RPC Post function.

`
POST /api/titles.transition_state HTTP/1.1
Host: api.example.com
Content-Type: application/json

{
"title": "123",
"status": "Out",
"user": "U123,U234"
}

`

By examples above, you can clearly see the benefits to design you API contract in a hybrid way. For resources, you can expose RESTful API to allow your API consumers to perform CRUD actions, and for the cases that requires background logic and data manapulation, the RPC approach is a better solution.

Why IDL matters

Now, let`s consider how to define a API contract. IDL (Interface Definition Language) is a must for developers to share the API specification. There are multiple reasons including:

1. IDL can be used to generate code artifacts.
2. IDL has no ambiguity.
3. IDL can be better managed in a VSC system (Github is my favorite).

I recommend protobuf as our IDL to define the API contract for reasons:

1. Protobuf can be used to generate the OpenAPI specification, whereas the OpenAPI cannot be used reversely.
2. Protobuf is widely used in the most popular RPC framework gRPC, which is actively developed and maitained by Google and community.