gRPC vs REST - Simple Performance Test

Overview

As a Java Developer, RESTful APIs are my bread and butter. On most days I spend a majority of my time writing code that either make up or interact with RESTful endpoints, or downstream code triggered by an endpoint.

I'd heard of some rumblings of a new RPC format a few years ago, and I immediately thought back to my days doing some light GWT (Google Web Toolkit) development. With that, I brushed the idea out of researching gRPC and continued with my standard workflows.

However, a few months ago I was given the opportunity to jump into a greenfield Golang project at work. I'd heard plenty of good things about Golang, so I dove right in.

I've since fallen in love with the language, though Java is still where I feel most at home.

However, while using Go I've come across gRPC usage more and more.

So as a part of my continued learning, I developed this simple little test to see how the gRPC vs REST performance debate actually plays out within Java alone.

This isn't an incredibly scientific experiment, but I feel like it at least provides somewhat clear results as to what one might expect when someone at your work starts pitching the new hotness (gRPC in this case) and swears that it will solve X or Y problem and have A or B performance benefits.

I've been that guy before, and it always helps to have relevant and concrete results to show everyone to help sway your point.

So with that out of the way, here's my silly little Java-based gRPC vs Rest performance experiment.

Rest vs gRPC tl;dr

Before we get into the experiment and testing, here's a quick overview.

RESTful APIs use HTTP 1.0/1.1 and include usage of HTTP Verbs (ex. GET/POST/DELETE) to transmit messages from client to server. Most servers will expose endpoints that expect specific headers and verbs and utilize the incoming data to perform operations. Data is commonly in JSON format, which Java code deserializes into Java objects for usage within the application.

gRPC on the other hand, utilizes HTTP/2 and protocol buffers, which can be simplified into binary over HTTP. The protocol buffers describe the length of the components in the binary stream. So rather than delineate data with JSON tokens (such as { , and " ), the data streams as binary and is parsed using known sizes of expected data types. This is then deserialized into Java objects.

Ok, there's an oversimplified overview, onto the test details.

Test Data and layout

The intention of this test was to give some samples of gRPC vs Rest performance, specifically. This is one of the more commonly cited reasons to switch to gRPC, as data is smaller as a binary stream than a json object.

This test was implemented using exactly 2 JVM applications. One is a Spring Boot application that contains RestController annotated classes to handle inputs and also contains a client that uses Spring Webflux's WebClient to make blocking API calls to the RestController.

The second application is a native Java application that uses io.grpc libraries to define a gRPC client and server.

These applications were deployed in two locations and run as runnable jars. Both were deployed on a local desktop as well as an AWS EC2 instance. Details for both are included in the next section.

The test is as follows:

• Use randomly generated data in a specific format that is identical in the gRPC and Rest testing

• Send increasing numbers of requests in order to see how the processing scales

• represent semi-real world performance by making calls from a local system to a datacenter at least 200 miles away.

Here is a sample layout of the Rest JSON template:

{
    "car": {
        "make": "toyota",
        "model": "corolla",
        "trim": "4DR Sedan",
        "color": "gray",
        "year": 2012,
        "mileage": 120000
    },
    "driver": {
        "firstname": "John",
        "lastname": "Smith",
        "driverId": "someId"
    }
}

Here is the proto definition for the gRPC messages

syntax = "proto3";
option java_multiple_files = true;

message Message {

    message Car {
        string make = 1;
        string model = 2;
        string trim = 3;
        string color = 4;
        uint32 year = 5;
        uint32 mileage = 6;
    }

    message Driver {
        string firstname = 1;
        string lastname = 2;
        string driverId = 3;
    }

    Car car = 1;
    Driver driver = 2;
}

These are functionally identical, and would be an example of what a transition from an existing Rest API to a replacement gRPC api might look like.

Test machines

The remote machine where the gRPC and Rest servers were running were t2.small EC2 instance machines. These were generic non-spot, non-dedicated t2.small instances that had all TCP and UDP traffic exposed. With a whitelisting for the local test machine that would be hitting it.

JVM Applications were run locally using the Windows 10 x64 OpenJDK, version 11.0.11. Remote JVMs were run using java-11-amazon-cordetto on a AWS Linux 2 OS.

The local test machine was my own personal development desktop. This system contains a Ryzen 3700x and 64GB of 3200ghz Memory. More than overkill for running the tests described above.

Results

Each test was run (from local to ec2) 10 times and the average taken. This were for sequential requests across Rest and gRPC as seen in the columns below.

There's a couple things we want to review as a part of these results. They are subsectioned below!

How much slower is Rest at each request amount

We can confidently say that gRPC will always be faster than Rest calls. This aligns with the implementation details around gRPC, specifically binary streams over JSON, binary parsing vs marshaling, etc.

What was the correlation between increase in requests and completion time for Rest and gRPC

Good question! So for each increase in requests, we would expect to see a linear increase in processing time to get through those messages. Ie, twice the messages should take twice as long. Let's see if that was actually the case.

Based on the results above, it's a bit hit or miss. Both Rest and gRPC roughly track the increase. Most of the time, the processing time increases at a faster rate than the request increases, which is good! It means we aren't exponentially taking more and more time to process more requests.

What does any of this tell us?

Outside of this specific test? Not a ton. It's a good example to use as a baseline, and a good simple experiment to compare the performance of the two paradigms.

As one would expect, the repository is available on Github with all source code.