Overview of My Submission
This submission is to switch my application (Sanic Currency Exchange API) from using PostgresDB to be able to use Redis Stack(RedisJSON) as data store, as shown in the diagram, it contains 3 parts of work.
- Refactoring, to implement Dependency Injection to split Database logic and Application logic.
- Auto Test/Regression Test, to guarantee that before and after db switch the API behaves the same, this part of work was done in Postman, and the test scripts were shared in repo.
- Switch Data Store to RedisJson, to implement a Redis repository which keep the same behaviour as Postgres repository, to make the data store switch happen easily and smoothly.
The Idea
I cloned this Currency API (Github link below) about 3 years ago and kept maintaining it. The initial thoughts were maintaining the dependencies to avoid some security problems. Gradually, I started to re-write the application codes to make it more maintainable, readable. (This is a long way to go, I am sure it has not been finished yet, 😀)
This API is to fetch currency rates from Europe Central Bank, and offer the rates in RESTful API.
Since I saw the Redis hackathon announcement, I got this idea to switch data store from Postgres to Redis JSON. I used Redis before as a caching layer for application, but this is my first time to use the Redis modules. This implementation probably have a lot to improve, please feel free to leave comments to make it better and help me to learn more about Redis JSON and Redis Stack modules.
Submission Category: MEAN/MERN Mavericks
- Use Redis to the RESTful API to add caching and advanced searching capabilities using JSON module.
- Use Redis as a primary database instead of Postgres.
- Redis Stack docker image was used here instead of Redis Cloud, to keep the Postgres and Redis Stack have the same hardware spec.
How the data is stored:
The data is store as a document, for each day of the currency rates.
{
"2020-01-01": {
"USD": 1,
"EUR": 1.1,
"AUD": 0.97
}
}
How the data is accessed:
In python codes, the data is accessed through a simple get
r.json().get('2020-01-01')
Language Used
- Python (Sanic Framework)
- Javascript
Link to Code
tim-hub / sanic-currency-exchange-rates-api
This is a self hosted, free, open source Python Currency Exchange Rate API fork.
Migrate Sanic Currency Exchange Rates Api to Use Redis Stack @Redis Hackathon
Use Redis Stack (RedisJson) to replace Postgres as main storage, plus some refactoring
How it works
How the data is stored:
The data is store as a document, for each day of the currency rates.
{
"2020-01-01": {
"USD": 1
"EUR": 1.1,
"AUD": 0.97
}
}
How the data is accessed:
In python codes, the data is accessed through a simple get
r.json().get('2020-01-01')
Performance Benchmarks
Load test is used here to measure the performance.
- same Machine (my personal macbook laptop)
- same spec for DB
- 10GB Volume
- Same ARM64 Instance
The 2 key results here are iterations
(higher is better) and iteration_duration(avg)
(lower is better)
Fact | Postgres | Redis Stack |
---|---|---|
Iterations | 25.26/s | 5.52/s |
Iteration Duration | 560.15ms | 5.53s |
This performance benchmark is surprising to myself as…
Additional Resources / Info
Collaborators
This is a solo submission.
- Check out Redis OM, client libraries for working with Redis as a multi-model database.
- Use RedisInsight to visualize your data in Redis.
- Sign up for a free Redis database.
Top comments (0)