Advanced Node.js Topics: Caching.

Improve your Server Performance By Caching often Fetched Data using Redis.

Introduction:

Redis is an in-memory database that stores data in key: value format, since it’s in memory, it's ridiculously fast.

Redis provides data structures such as strings, hashes, lists, sets, sorted sets

Redis is an open source (BSD licensed), in-memory data structure store used as a database, cache, message broker, and streaming engine. Redis provides data structures such as strings, hashes, lists, sets, sorted sets with range queries, bitmaps, hyperloglogs, geospatial indexes, and streams. Redis has built-in replication, Lua scripting, LRU eviction, transactions, and different levels of on-disk persistence, and provides high availability via Redis Sentinel and automatic partitioning with Redis Cluster. redi.io

case use:

Why you should implement caching layer in your server and how it will improve the performance of the application.

• Regular fetched data
  caching data that gets fetched too often is a good practice, you won’t let users wait too long to get basic data

• Reduce query exec against the database
  by using caching you will reduce the number of queries that will be executed against the database, aside from the performance improvement you’ll get, you also save your database from being overwhelmed with duplicate queries that return the same result.
  also, you will save your precious bandwidth if you are hosting your app at an expensive hosting provider

• Improving the app performance
  caching will not only improve your database performance and protect it, but it will also improve the overall server-client performance.
  to get the idea let's assume your server has a route called getArticle, every request that comes to this route will take about half a second (~500 ms ) to get the response, with caching the first request will take about half a second or more, but every next request will take about (~20 ms)! Magic right ?!
  I’ll prove it using my own app

Cache vs no Cache:

I did a simple benchmark test on my server (on a single API to see how it takes to complete the request without caching, and re-test the same API with caching to see the improvement.)

the above image shows you, that it took my server more than a second to complete the request, of course, every time I make a request it will take approximately the same time!

The below image is when I introduced the Caching mechanism to my server’s APIs, you can see the difference in the time by yourself ( i did not change anything in the request)

the time to complete the same request in the upcoming times has been reduced to almost ~10 ml, which is a significant improvement!

I hope now you're excited about implementing this technique in your own projects.

Installation:

Install Redis on your machine

in Linux, you can install it by typing in the terminal

sudo apt-get -y install redis

in macOS, in terminal issue the following command

brew install redis

in windows, it's a little bit difficult to get Redis on windows since it’s not officially supported in widows.

Redis is not officially supported on Windows. However, you can install Redis on Windows for development by following the instructions below.
To install Redis on Windows, you’ll first need to enable (Windows Subsystem for Linux). WSL2
WSL2 lets you run Linux binaries natively on Windows. For this method to work, you’ll need to be running Windows 10 version 2004 and higher or Windows 11.

https://redis.io/docs/getting-started/installation/install-redis-on-windows/

Install node-Redis to your project:

node-Redis is a modern, high-performance client for Node.js.

npm install redis

Now we installed both Redis and node-redis package, let's do simple work with these great tools and then try it in a real-world example!

Quick start with Redis:

// IMPORTANT : before you can establish connection to redis,

// you must start the redis-server

const redis = require('redis');

// create client to communicate redis DB

var client = redis.createClient();

client.connect()

// connect and error events

client.on('error', function (err) {

console.log('Something went wrong ', err)

});

client.on('connect', function () {

console.log('Redis Connected!')

});

connect to Redis server:

To start the Redis server you need to run in the terminal :

redis-server

example:

Above we created a client and started the connection to Redis server, now we can use all the features.

// used to put data into redis
client.set("key" , "some value")

// used to get data from redis
client.get("key")
// output = "some value"

Real-world example:

the big question now is how can I make use of these simple functions set() and get(), to improve my server performance?

let’s see my own server (express server to fetch quotes from MongoDB atlas and send it back.)

This Benchmarking test is from my own Quote API (which will be released soon in RapidAPI), we’ll see how the difference in the time it takes when we request 1000, 3000 and 5000 documents of data, I’ll repeat the test by fetching the same data but from the cache this time.

to fetch 1000 documents from the DB took almost 2 seconds

But fetching the same amount of data from the cache it only took 25 milliseconds!

to fetch 3000 documents from the DB it took almost 4 seconds!

But fetching the same amount of data from the cache it only took 45 milliseconds!

to fetch 5000 documents from the DB took almost 5 seconds!

But from the cache, it only took 60 milliseconds!

Pretty amazing right??

The caching mechanism:

caching is simply adding another layer to your server, this layer will intercept the queries that will be executed by your database, it will search if this query is cached before or not, if so it will return the cached data as a response and will not going to send the query to the database, if the query has not been cached before, it will send the query to the database to get executed and then store the result in the cache (Redis) for the upcoming requests.

So to clarify what we want to do :

1. connect the server to Redis using the above code

2. we want to hijack (intercept, interrupt, call it whatever you want) the process of sending the query to the database so we can decide if this query is cached before or not

3. if cached, return the cached data and end the response. Don’t send anything to the database

4. if not cached, send the query to get executed and send the result as the response, then store the result in the cache for the new requests.

First of all, you need to create a new file in your project called cache.js, you can name it whatever you want, in the services directory if you have one, if not just put it anywhere.

this file will contain all the logic needed by our server to cache data and retrieve data from Redis.

In the beginning, we’ll need to connect to Redis and make sure it's working fine

key creation:

to store data in Redis we need to give every query a unique and consistent key, so we can retrieve the right query when a request has arrived.

basically, every query to any database has filters and options to get particular documents or records.

we’ll take advantage of this thing and take these filters and turn it into a string

 JSON.stringfy({ {title : "story"} , {skip : 10} })
    // our key = "{{"title" : "story" }, {"skip" : 10}}"
    // everytime any client will request the data that can be fetched
    // with this query, the server will repsonse with the cached data

Now we have our key for every query need to be executed in the database, what we’ll do next is to search in Redis for this key, if it does exist, return its value instead of executing the query, if not found, execute the query in the database and store the query result with its key in Redis for next requests and then send the result to the user.

// this line will add the data fetched from mongoDB to redis
client.set(key, JSON.stringify(result))
// always remember, Redis only store values as a string or numbers

Wrap up:

first, you need to create a unique and consistent key for every query.

second, search in Redis for this key, if it's found, return its value as a result, if not, execute the query and store the result in Redis and lastly send it to the user.

References:

The best of all courses to learn node.js in depth.

https://www.udemy.com/course/advanced-node-for-developers/?ranMID=39197&ranEAID=SAyYsTvLiGQ&ranSiteID=SAyYsTvLiGQ-EDfB120pgYcJlhkNSEBp4Q&LSNPUBID=SAyYsTvLiGQ&utm_source=aff-campaign&utm_medium=udemyads
How To Implement Caching in Node.js Using Redis | DigitalOcean
Redis + Node.js: Introduction to Caching - RisingStack Engineering
Caching in Node.js using Redis
Fasten your Node JS Application with a Powerful Caching Mechanism using Redis

Redis OM for Node.js

Thanks for reading, and feel free to ask any question about javascript or this series, I appreciate any feedback to improve My content.

find me on Twitter, Github and my portfolio.