DEV Community ๐Ÿ‘ฉโ€๐Ÿ’ป๐Ÿ‘จโ€๐Ÿ’ป

Cover image for The in-memory & cache DB (Amazon ElastiCache)
Augusto Valdivia for AWS Community Builders

Posted on • Updated on

The in-memory & cache DB (Amazon ElastiCache)

โœจ Wow! Give yourself a tap on your back for going this far.

We have learned about 5 different amazing ๐Ÿ˜ฎ AWS databases throughout this series of articles. Please stay with me until the end of this series, I do not wish to spoil the grand final ๐Ÿ˜‡ when we will finally build a great mini project along with one or two of these databases.

Summary of what we have learned so far:

On this article we will learn about our next database Amazon ElastiCache

What is an Amazon ElastiCache?

Amazon ElastiCache is a fully managed in-memory data store and cache service by Amazon Web Services. The service improves the performance of web applications by retrieving information from managed in-memory caches, instead of relying entirely on slower disk-based databases.

RAM

How does one know when to choose an Amazon ElastiCache?

If you are searching for a database that allows you to query your data using keys with microsecond latency, do not look further Amazon ElastiCache is your DB of choice. A particularly important fact to take into consideration about Amazon ElastiCache is that it supports the Memcached and Redis cache engines. As we continue to the end of this read, I will provide you with some advantages of using one or the other. Hoping that this information will help you to choose the right engine and version that best meets your requirements.

What is some industry Amazon ElastiCache use cases?

  • Real-Time Analytics
  • Caching
  • Leaders Boards
  • Mobile
  • Product catalog
  • News
  • Tickets seller
  • Need for speed(submillisecond latency)

Now that we have a general idea of Amazon ElastiCache let us continue drilling deeper and let us explore and build an Amazon ElastiCache resources:

First thing first โ€“ When to use Memcached vs Redis?

Memcached:

  • Require the simplest model possible
  • Require running large nodes with multiple cores or threads
  • Require the ability to scale out and in, adding and removing nodes as demand on your system increases and decreases
  • Require caching objects

Redis-Versions might apply:

  • ElastiCache for Redis version 6.2 (Enhanced) ability to tier data between memory and SSD using the r6gd node type
  • ElastiCache for Redis version 6.0 (Enhanced) authenticates users with role-based access control
  • ElastiCache for Redis version 5.0.0 (Enhanced) Redis streams, a log data structure that allows producers to append latest items in real-time and allows consumers to consume messages either in a blocking or non-blocking fashion
  • ElastiCache for Redis version 4.0.10 (Enhanced) encryption and dynamically adding or removing shards from your Redis (cluster mode enabled) cluster

Note: I did not include all Redis versions available on this list. For more information, I will be adding a reference link to this page. I hope this comparison has given you an idea of which engine to choose for your future projects.

Now that we have a general idea about these 2 engines let us continue exploring more details and build them using Terraform:

Let us see some important characteristics about Amazon ElastiCache

Memcached


Performance: 

- Automatic detection and recovery from cache node failures
- Automatic discovery of nodes within a cluster enabled for automatic discovery, so that no changes need to be made to your application when you add or remove nodes
- Integration with other AWS services such as Amazon EC2, Amazon CloudWatch, AWS CloudTrail, and Amazon SNS to provide a secure, high-performance, managed in-memory caching solution
- Each cache node has its own Domain Name Service (DNS) name and port

Scalability: 

- Every node runing an instance of Memcached can scale the nodes in a cluster up or down to a different instance type
- Memcached supports up to 300 nodes per customer for each AWS Region with each cluster having 1โ€“40 nodes
- More than 40 nodes in a Memcached cluster, or more than 300 nodes totalFill out the ElastiCache Limit Increase Request form at: https://aws.amazon.com/contact-us/elasticache-node-limit-request/

Availability:

- Flexible Availability Zone placement of nodes and clusters
- Amazon ElastiCache for Memcached is available in multiple AWS Regions around the world
- It's recommended to use consistent hashing while partitioning data
- When you scale your Memcached cluster up or down, you must create a new cluster. Memcached clusters always start out empty unless your application populates it

Security: 

- All new ElastiCache clusters are launched in an Amazon Virtual Private Cloud (Amazon VPC) environment
- ElastiCache node access is restricted to applications running on whitelisted Amazon EC2 instances
- Access to cluster outside a VPC is controlled by security groups
- AWS is responsible for protecting the infrastructure that runs AWS services in the AWS Cloud
- You are also responsible for other factors including the sensitivity of your data
- Your companyโ€™s requirements, and applicable laws and regulations
Enter fullscreen mode Exit fullscreen mode

Redis


Performance: 

- Redis (cluster mode enabled) supports partitioning your data across up to 500 shards.
- Integration with other AWS services such as Amazon EC2, Amazon CloudWatch, AWS CloudTrail, and Amazon SNS. This integration helps provide a managed in-memory caching solution that is high-performance and highly secure.
- ElastiCache for Redis manages backups, software patching, automatic failure detection, and recovery.
- You can have automated backups performed when you need them, or manually create your own backup snapshot. You can use these backups to restore a cluster. The ElastiCache for Redis restore process works reliably and efficiently.
- Data tiering provides a price-performance option for Redis workloads by utilizing lower-cost solid state drives (SSDs) in each cluster node in addition to storing data in memory. It is ideal for workloads that access up to 20 percent of their overall dataset regularly, and for applications that can tolerate additional latency when accessing data on SSD.

Scalability: 

- Automatic detection of and recovery from cache node failures
- You can get high availability with a primary instance and a synchronous secondary instance that you can fail over to when problems occur. You can also use read replicas to increase read scaling
- Redis (cluster mode enabled) clusters can have up to 500 shards

Availability: 

- Multi-AZ for a failed primary cluster to a read replica, in Redis clusters that support replication
- Flexible Availability Zone placement of nodes and clusters for increased fault tolerance
- By using the Global Datastore for Redis feature, you can work with fully managed, fast, reliable, and secure replication across AWS Regions. Using this feature, you can create cross-Region read replica clusters for ElastiCache for Redis to enable low-latency reads and disaster recovery across AWS Regions

Security: 

- For Redis version 3.2 and later, all versions support encryption in transit and encryption at rest encryption with authentication. This support helps you build HIPAA-compliant applications
- You can control access to your ElastiCache for Redis clusters by using AWS Identity and Access Management to define users and permissions
- All new ElastiCache clusters are launched in an Amazon Virtual Private Cloud (Amazon VPC) environment
- ElastiCache node access is restricted to applications running on whitelisted Amazon EC2 instances
- Access to cluster outside a VPC is controlled by security groups
- AWS is responsible for protecting the infrastructure that runs AWS services in the AWS Cloud
- You are also responsible for other factors including the sensitivity of your data
- Your companyโ€™s requirements, and applicable laws and regulations

Enter fullscreen mode Exit fullscreen mode

Please note that some of these limitations are adjustable and others are not.

Oh, right it is building ๐Ÿ‘ท time let us build this database using Terraform

DBS

Terraform code previous:

resource "aws_elasticache_cluster" "memcached_engine" {
  cluster_id           = "cluster-memcached"
  engine               = "memcached"
  node_type            = "cache.m4.large"
  num_cache_nodes      = 4
  parameter_group_name = "default.memcached1.4"
  port                 = 11211
}

resource "aws_elasticache_cluster" "redis_engine" {
  cluster_id           = "cluster-redis"
  engine               = "redis"
  node_type            = "cache.m4.large"
  num_cache_nodes      = 2
  parameter_group_name = "default.redis3.2"
  engine_version       = "3.2.10"
  port                 = 6379
}


Enter fullscreen mode Exit fullscreen mode

Find the Terraform repo and directions for this project here

Diagram

Ecached

References:


https://docs.aws.amazon.com/ElastiCache/latest/developerguide/limits.html 

https://en.wikipedia.org/wiki/Amazon_ElastiCache

Enter fullscreen mode Exit fullscreen mode

Top comments (0)

Timeless DEV post...

How to write a kickass README

Arguably the single most important piece of documentation for any open source project is the README. A good README not only informs people what the project does and who it is for but also how they use and contribute to it.

If you write a README without sufficient explanation of what your project does or how people can use it then it pretty much defeats the purpose of being open source as other developers are less likely to engage with or contribute towards it.