Aerospike REST client

It is fairly well known that Aerospike is synonymous with speed and ultra low latency. While our server holds this badge, many a times, during end-end application development there is this myth that some of our clients are slow especially those written in interpreted languages Python/Ruby etc.

Now, I am a huge fan of Python and it's my go to language for any web application. So I am bound to defend this and also demonstrate that you don't actually have to use a language specific client. Aerospike offers a REST client. Technically this is a full fledged backend, the name is kind of a misnomer as it runs a Spring Boot application under the hood.

Let's take a typical use case - Push JSON data to a datastore. For example this is the data from my front end/mobile app that I need to persist.
{"name": "xyz", "id", 23234, "email": "blah@blah.com"}

The traditional way to do this is to write a backend or use one of the serverless frameworks or use a platform such as Firebase.

I have written a simple web backend using a state of the art Python web framework FastAPI that stores this JSON into Aerospike using a key.

This took less than 10 minutes to write but a better alternative is to just use the REST client. This is why:

• Simplicity - You just run a jar file and that's it.
• Full suite of APIs - The REST client is on par with all other client libraries.

Ok, with that, let's do a simple benchmark.

Setup

• Install the Aerospike python client. pip install aerospike.
• Run Aerospike db as a Docker container on interface 0.0.0.0. Here is my previous blog post on how to accomplish that
• Download the latest Aerospike REST client
• Have Java 8 or higher on the machine and we are all set.
• Download and install Apache bench

Comparisons and benchmarks with AB

The Python FastAPI server

from fastapi import FastAPI, Request
import aerospike

app = FastAPI()

# Aerospike connection client
config = {
    "hosts" : [('0.0.0.0', 3000)]
}
client = aerospike.client(config).connect()

@app.get('/v1/kvs/{namespace}/{set_name}/{key}')
async def get_data(namespace, set_name, key):
    """
    The API signature matches the Aerospike REST client's
    """
    user_key = (namespace, set_name, key)
    key, metadata, record = client.get(user_key)
    return {"key": user_key, "record": record} 

@app.post('/v1/kvs/{namespace}/{set_name}/{key}')
async def post_data(namespace: str, set_name: str, key: str, request: Request):
    data = await request.json()
    user_key = (namespace, set_name, key)
    client.put(user_key, data)
    return {"message": "success"}

Running the REST client
java -jar as-rest-client-1.5.0.jar --aerospike.restclient.hostname=0.0.0.0'

AB results

Uvicorn Benchmark Screencast
REST Benchmark Screencast

Not surprisingly both of these have similar benchmarks about 2700 rps. One might run Uvicorn with just 1 worker thread and dismiss Python as slow, but that's not correct. To compensate for Java's multithreading the least we can do is run as many workers as the number of cores. Since my machine has 8 cores I chose 8 workers and the results are more or less the same.

One key difference is that because we are starting multiple processes each of these will open a separate connection to Aerospike. The issue with this is that each process has a client, each client opens its own connections. Without tuning, those connections can swamp the server.

It's far more efficient to share fewer clients, as the REST server does. In fact, this was one of the guiding principles behind developing the REST client.

In addition, the amount of time spent on writing the Python application is totally saved if you choose the REST client route.

I've found the REST client to be a great alternative to writing your own backend for storing data in Aerospike.

The REST client also offers predicate filtering abilities out of the box which is very helpful. A two part blog series on that topic can be found here and here

Next steps

An Aerospike Docker container with the REST client. This all in one container will be ready to use and saves all the Java and dependencies installation time.