When you run after the horse in the rabbit whole

Welcome dear reader, I am Silent-Mobius, AKA Alex M. Schapelle, your faithful servitor that continues to lead with examples of learning of new topics with trials and errors.

As mentioned in our previous encounter, I have started to dwell into world of databases, by exploring two main types which are NoSQL and RDBMS, mainly talking about MySQL and Redis. About why I am doing this, we'll explore later, as on this entry we'll mostly discuss the architecture as mentioned before and according to table of content:

Let create a small diagram that will illustrate what we are going to create:

We'll be using docker containers to pack our application, and deploy the packed images into container registry. Within Gilab-CI we'll work on our code, and in case it will pass, it will start new pipeline that will deploy on test network with docker compose the scaled application, Redis-cluster and PostgreSQL-cluster.

There will be configuration files for Redis to store data with SQL server and in term, SQL server with be saving data to volume that will be saved under backed up volume... it seems there will be a lot of configuration to be done, either with Python, Bash or Ansible.

The architecture is inspired by Alex Xu book named System Design Interview

As we can see there are a lot of things to setup, create, manage, provide and deploy, thus here is a list of things that I am going to use:

• Language: Python, mostly due to the development can be fast and in this case I'd prefer to focus on learning DB and Cache part more then learning new programming language.
• Version Control: Git, the application is not going to be that big, in addition today git is the de facto the best choice for code version control, which is also incorporated within CI
• CI : Gitlab-CI, Because my github account got locked and the support suggested to destroy the account but not open it for me, thus they lost me as a customer.
• RDBMS: PostGreSQL, mostly to master the automation of main and secondary installation with Python as well as auto-database setup or connection with new instance.
• NoSQL: Redis, to learn the caching mechanism provided by Alex's book
• Micro-Services: Docker and Docker Compose, for only development purposes, eventually we'll create k8s deployment

Although I've been working as IT professional for 15 years, and have been working on these systems separately, It never occurred to me that I need to connect them by myself, mostly because we used to work with waterfall methodology and guilt, along side new application version was dropped down at us to handle by development team. Our Dev team were cool guys and were always there to help, but lack of knowledge and experience of high-level programming languages made it very hard to determine to whom we should consult for aid.
When idea to learn enough programming to do tasks popped up, though it was there for some time, with it came uncertain choice:

• Perl
• Ruby
• Python
• Node.js

Which one to use ?

I chose Ruby which in long run, was somewhat incorrect choice, and moving to python from it was not hard but a very long and confusing journey.
Today I stumble to same dilemma, whether to use Go or Rust, to which I've chosen Go... but this is for another article.

All the other automation I've done, was mostly with shell or Bash, with Linux Built-in tools, which eventually turned into Cloud-init, Puppet andAnsible`, thus burdening myself with SQL or NoSQL, was never the issue for me, to what I somewhat regret.
Thus this is where I start to learn ... and hope you will learn with me. No matter what, Please Do Try To Enjoy

Till next time