In this blogpost I would like to present you a possible implementation for a batch route using FastAPI. FastAPI is a great framework for creating REST API's, unfortunately there is currently no native support for batching.

Why batching?

Batching is the process of combining multiple API requests into a single request with a single response. Without batching, the whole thing looks like this:

Each request between client and server has a certain network latency, so the processing of several consecutive requests can take some time.

With a batch request, the roundtrip between client and server and thus the number of requests can be reduced, which in turn rapidly improves performance:

Technical consideration of the batch route

In the course of my research, I looked at the batch routes of other major API's, including:

• Microsoft Graph API Batch Route
• Microsoft Sharepoint API Batch Route
• Google Cloud Storage API Batch Route

Based on the API's listed above, there are two different ways to submit batch requests:

• Submit batch request with MIME type multipart/mixed
  • The advantage of this approach is that requests can be sent with different MIME types (e.g. application/json, application/http, application/octet-stream etc.).
  • The body of the request is divided into parts that are separated from each other by a boundary string that is specified in the header of the request.
• Submit batch request with MIME type application/json
  • The batch request is submitted in JSON format. This limits the API to the MIME type application/json.

I personally prefer the JSON batch approach, assembling the batch request is slightly easier with this approach. Of course this approach is limited to the MIME type application/json only, but since most FastAPI routes implement exactly this MIME type this is acceptable for me. The choice therefore varies simply on the technical requirements.

Our batch route is therefore inspired by the Microsoft Graph API, the batch request will have the following structure:

POST /batch HTTP/1.1
Content-Type: application/json

{
  "requests": [
    {
      "id": "1",
      "url": "/products?skip=0&limit=1",
      "method": "GET"
    },
     {
      "id": "2",
      "url": "/products/1",
      "method": "GET"
    },
    {
        "id": "3",
        "url": "/products",
        "method": "POST",
        "headers": {
            "Content-Type": "application/json"
        },
        "body": {
            "title": "Test Product"
        }
    }
  ]
}

The next step is to consider how the individual requests should be processed on the server side. Behind each FastAPI route there is a corresponding Python function that can be called directly. At first this sounds like a good idea, but then FastAPI is no longer able to determine the dependencies via depency injection, so we would have to do this manually.

Determining the dependencies manually could be difficult and probably often leads to unexpected behavior of the API route. So I did some research and came across this repository:

pip install Flask-Batch

# to include the dependencies for the batching client
pip install Flask-Batch[client]

from flask import Flask
from flask_batch import add_batch_route

app = Flask(__name__)
add_batch_route(app)

# that's it!

from flask_batch.client import Batching
import json
alice_data = bob_data = {"example": "json"}

with Batching("http://localhost:5000/batch") as s:
    alice = s.patch("/people/alice/", json=alice_data)
    bob = s.patch("/people/bob/", json=bob_data)

alice         #

Flask-Batch performs batching on the HTTP layer. This means that on the server side a HTTP client executes all batch requests. The network latency is much lower on the server side which implies that the requests can be processed faster on the server side. Additionally, since the requests are processed on the HTTP layer, FastAPI is also able to resolve the dependencies via dependicy injection. Therefore I also decided to do batching on HTTP level.

Let's get started

Defining the batch models

HTTPVerbs

This enum defines the allowed HTTP methods.

BatchRequest

This model represents a single API request. This model also ensures that the referenced endpoint meets our requirements, in this case that no absolute URL is given and that the route starts with a leading slash. Especially the check that it is not an absolute URL is extremely important to prevent potential improper usage of our batch route.

BatchResponse

This model represents the result of the corresponding request. Here also the ID of the request is returned so that the client can map request and response (theoretically the order of processing could vary). Besides the ID, the HTTP status code, the headers and the JSON body are also returned.

BatchIn

This is the container that includes all our batch requests. Here it is also checked if all ID's are unique and the allowed maximum amount of requests has not been exceeded.

BatchOut

This is the container that holds all the responses, this model is the one that is passed back to the caller.

Implement the batch route

The batch requests are processed using the Python package aiohttp. This allows the requests to be processed in an asynchronous manner. One blog post that helped me a lot with the implementation is the post Making 1 million requests with python-aiohttp which I would definitely like to recommend to you!

In the next step we implement the actual batch route, here we just have to plug everything together:

The repository, which contains all the code including a small test API, can be found here:

sequenceDiagram
 autonumber
 Client ->> Server: API Request
 Server ->> Client: API Response
 Client ->> Server: API Request
 Server ->> Client: API Response
 Client ->> Server: API Request
 Server ->> Client: API Response

Conclusion

Bathing is an important component to combine many requests to a single API call and thus increase the performance rapidly.
I hope you enjoyed my blog, thanks for reading! If you have any recommendations on how to improve my implementation, please let me know in the comments section.