Explained with code example - What is API statelessness and why do we care?

We’ve all heard about “statelessness”. For a while, I just took it for granted, as a thing you can do when designing interactions between a client (for simplicity say a frontend end-user, but this is not necessary) and a server. Statelessness is one of the core tenets guiding RESTful API design.

Recently though, I learned some material which really drove the idea home for me.

Also, being hand-wavy sucks, so we will follow a concrete code example to drive home the point.

Our reference example

We will mock up a SUPER basic client and server interaction. The client will be fetching for a list of animal names by calling out to a “backend” (quotations because it’s so simple backend seems a stretch).

Except, we will only fetch 5 animal names at a time, with the idea being that we can fetch more (unique, i.e. not repeating animals that were previously fetched by the client) animal names with each separate request. You’ve definitely seen this functionality in action before in some way, scrolling to the bottom of any social media feed for example (pagination).

Why might we want such a feature? Why don’t apps send over all data at once?

While a text list of a few animals is an insignificant amount data to query or send over a network, it becomes a very different equation when, for example, sending high quality images (which can go to 2MB) and even more so for videos of course. Trying to send this stuff would severely slow the response (i.e. increase latency) and increase the workload for both the server and the client devices, making the overall interaction pretty bad.

What does it mean to be stateless?

A standard client - server interaction

A client communicating with a server for info, we all know this pattern. The client sends a request to the server, the server responds with some response. There are of course many different ways of going about making this concrete, there’s different protocols (HTTP, WebSockets, WebRTC) and frameworks to implement those protocols in action (REST, GraphQL, gRPC). But the basic remains consistent.

💡 This interaction can be called stateless, when the server does not store any user specific information on itself that it will then use in any way to guide its response to the client’s request.

This does not mean that the server is not interacting with or storing content in a data store. In fact, servers could cache certain popular requests for themselves to save on lookup time and costs. Instead, the idea is that if you replicated this backend server from scratch, if the next client request instead went to a different server, the server would be able to service the request adeptly without any hitches, and specifically, without having to coordinate or sync any info from the last server which serviced this client.

A client interacting with multiple server instances

Why might we want our APIs to be stateless?

Here’s four central reasons:

1. Statelessness reduces complications when scaling servers

   Pretty much for the exact scenario laid out in the diagram above, when scaling (horizontally), stateless APIs enable servers to individually and independently handle client requests, without coordinating with the other servers in the cluster. Thus, this enables inter-server independence and the ability for certain sets of servers to evolve or transition independently.

   Otherwise, we would need to set up some processes that would allow for servers to sync this user-specific context info between one another. Complexity abound, and some questions are posed if trying to accomplish this: What if a server crashes before or in the middle of performing this sync? How much of the servers’ resources will be overtaken by this syncing procedure? If we are using a third server to hold user session relevant data, will we need to think about that server's scalability as well?

   Simplicity is often the best policy, and statelessness will enable this for us.

2. Stateless isolation of client and server allows for modularity and flexibility.

   A small point, but an important one none the less. Isolating client and server allows for the clients and servers to be less co-dependent on one another, especially when it comes to more complex logic implementations. Each section can be swapped out and/or evolve individually without much overhead about how the network interactions between the components will handle the change.

3. Stateless implementations assist in caching, both client and server side.

   Implementation of statelessness often just results in a client sending all the necessary info required to fulfill a request, with the request itself - albeit ensuring CSRF and query string info exposure vulnerabilities minimized as guided by OWASP.

   This can be done with HTTP GET requests via either of url params (google.com/:add_option_here/) OR query params (google.com/?searchParam=myparam). Requests of course can also be sent via other methods, like via an HTTP POST using body params.

   Think of a basic HTTP GET request (fetching info). In a stateless Client-Server API format, a the client and server can be confident that unless certain circumstances have changed, a request will be idempotent, i.e. barring the change of certain pre-definable conditions, the request will render the same response between two queries. Therefore, this will allow a client and/or server to cache certain query responses so that time and resources are saved before a query is made and/or to respond to a query.

   
   An example of a simple cache layout. Diagram referenced from: https://muthu.co/client-side-caching-of-api-responses/

4. Stateless implementations REST APIs are easier to test.

   Working forward from the previously covered benefit that stateless REST APIs are idempotent, we also derive that these API formats enable for the implementation of pure functions, i.e. functions wherein the return value is only determined via its input arguments without side-effects. The benefit of this format of function endpoints is that they align well to functional testing protocols, and therefore test suite implementations covering these functions are easier to implement.

Drawbacks and the alternatives to statelessness:

Naturally, stateless API implementations are prone to certain consequences, the big one centrally being that it restricts (or complicates) key functionality that web-apps want.

Any remotely complex web-application will require user-session tracking in at least some fashion anyways.

Think user authentication! That's a session token that is delivered from the client to the server on each request and is validated via middleware before being let access to most endpoints. This is a client-server session based information that will indeed need to be temporarily persisted on a shared data store (like Redis), and will also need to be accessible by all the server instances that are handling requests.

Therefore, such a simple user functionality will already require system designers to evaluate setting up an accessible 3rd server data store. More complex web-apps will want for much more involved functionalities that require more robust and scaled user-session info stores. Engineers have to tackle questions and corresponding solutions like:

• How do you minimize latency? Well there's plenty of ways to go about this. Centrally you want to cache user-session info, likely on a temporary 3rd machine data store like Redis, in an easily server-accessible manner.

  You may want to optimize hits to the right data-store cache via methods like data sharding and replication. Basically here you would orient certain set of server requests repeatedly to the same data-store cache to optimize cache hits and therefore reduce lookup time and latency. Also, depending on the value/criticality of the user-session info, you may elect to stand up backups of the session data using a service like AWS ElastiCache.

As is hopefully understandable now, solutions can indeed be engineered to allow for thriving user-session info retrieval by web servers. They do however undeniably add technical overhead.

Example Scenario:

So in our example, we want to implement this function of fetching 5 new animals from our backend server(s). We have two ways of implementing this:

1st method (stateful):

A user will be assigned a session ID by the server. The server for its part will tab this session ID alongside any user specific information it wishes to keep (like which index of queried animal the user is on) within an in-memory data store, for the sake of this example, say Redis (to allow for swift lookup).

Meanwhile the client will store this session ID via use of some local store. Each time the client sends a request to fetch 5 additional animal names from the server, the server will be able to easily look up the user via their provided user ID, fetch the user’s current session stored info, and return 5 new animals that specifically begin from the index till (not including - with 0 indexing) index + 5, which will then be returned to the user.

You can clearly see how this strategy has its pros and cons. On one hand, the server is able to store user session specific information in a quickly accessible fashion. On the other hand, a data store holding user session info is clearly not scalable, as multiple servers will require access to synchronized session stores. Here we may need to pull out some of the solution-ing discussed above.

Code implementation: Here is the architectural overview of how I went about mimicking this layout in a very simple fashion.

As seen in the diagram, for my implementation, I leveraged Postman to 'mimic a frontend' HTTP request. I implemented user sessions via the express-session NPM package, while using a local (literally on my computer) Redis instance to store this user session state, specifically utilizing the redis and connect-redis NPM packages. As for the animal names themselves...I saved myself the trouble: I asked chatGPT to produce 42 unique animal names and stored it as an exported JS array.

If curious, you can see the code for this setup alongside the relevant Postman scripts at my GitHub repo.

2nd method (stateless):

In this implementation, the client itself (say a simple react app) will handle the current index state on the client side (i.e. the frontend)! It will then pass this information to the server via url query params, which the server will then utilize to guide its response.

Here’s the HTTP endpoint layout for this request: /animals/?index=5

As the client will handle the state, the servers are free to be multiplied for horizontal scaling without any fear about state sync.

Clearly however, there are negatives to this solution as well - namely that this is an absurdly simple use case and we are not leveraging or collecting any valuable user session information for functionality.

Code implementation: Here is the architectural overview of how I went about mimicking this layout

Won't lie, this set up was dead easy. As seen in the diagram, for my implementation I still used Postman for HTTP calls, and backend was a simple express endpoint referencing the previously mentioned JS array of animal names and some error checking on the request. That is all and I am free to scale to more express servers without any technical overhead - as long as I am assured that the client is handling state effectively.

If curious, you can see the code for this setup alongside the relevant Postman scripts at my GitHub Repo.

Talk is cheap, show me the code.

Taking this as cue from Linus Torvalds, here we are. You can find the full code for this demo, including Postman scripts, on my GitHub repo. Also, here's a video of the demo: YouTube Demo.

And here's an overview of the layout:

To demo a model of the entire system put together, I leveraged Nginx (How to set up on Mac) as a loadbalancer and proxy, listening on port 8081. My Postman scripts only communicated with this Nginx server.

I spun up two instances of the stateful express app (on two different ports), and two instances of the stateless express app (on two different ports). I then guided HTTP get requests from Postman, such that /stateful/animals requests were guided to either of the stateful express app ports, with one being weighted more than the other (weighted loadbalancing), and exactly similarly for the /stateless/animals/animalIndex=X requests to the stateless express app ports.

Here's some helpful Nginx instructions (verified for Mac):

• If you install Nginx with Homebrew, your nginx config is located at /opt/homebrew/etc/nginx/.
• Here's how you run Nginx with a custom config: nginx -c /path/to/file/nginx_expressTest.conf.
• If you wanted to check where your default Nginx config is located, run nginx -t.
• If you wanted to see whether there was an Nginx process up and running, run ps -ef | grep nginx

You can see my sample Nginx config below:

# model nginx.conf for proxying and further loadbalancing between two express backends
events { }

http {
    # first defined upstream wherein there is a weighed loadbalance between two (stateful) express backends
    upstream express-backend-stateful {
        server localhost:3000 weight=3;
        server localhost:3001;
    }

    # second defined upstream with a simple proxy to a single (stateless) express backend
    upstream express-backend-stateless {
        server localhost:3002 weight=3;
        server localhost:3003;
    }

    server {
        # Nginx server to listen on port 8081
        listen 8081;

        # first defined proxy for the stateful express backend
        location ~ ^/stateful(/.*)$ {
            # '$1' is needed to pass the remainaing path (i.e. '/animals') to the proxied backend
            proxy_pass http://express-backend-stateful$1;
        }

        # second defined proxy for the stateless express backend
        location ~ ^/stateless(/.*)$ {
            # '$1' is needed to pass the remaining path (i.e. '/animals/?animalIndex=X', where X is a positive integer) to the proxied backend
            # 'is_args' and 'args' are needed to pass the query string to the proxied backend
            proxy_pass http://express-backend-stateless$1$is_args$args;
        }
    }
}

What's key about this implementation, and something I mentioned in my demo video, is that the stateful version of the setup works in this instance thanks to a shared local Redis instance which both the express servers had access to. Servers running on separate machines with individual Redis instances would need to worry about session data synchronization, and servers sharing a Redis instance on another machine would need to tackle latency.

Was this code too complex to drive home a simple example? Probably. But it was fun.

Hope ya got something from reading this.