Storage 101

Storage

What is Storage and why is it important.
In the last past 5 years, we have created more data than we have done in the last 100 years and the trend is expected to grow significantly and storage are the only way for us to store and keep those data, these show how critical storage technologies are in the life of any organization.
It is commonly said that data is now the most valuable commodity on earth if so, storage is also by default one of the most important piece of tech that any company hopping to compete in the digital age must deeply master.

Storage related terms such as Durability, Availability and redundancy must be fully grasped. If fact storage is one of the few components that constant grows during the lifespan of a company and most be closely monitored and secured.
Let’s get started with the main two type of Storage: Volatile Storage (AKA Memory) and non-Volatile Storage.
Volatile storage is in laymen term is the storage that goes away when a computer of a server is rebooted. A good example of volatile memory is RAM (Random Access Memory). To learn more about RAM, check my other blogpost here.

Non-Volatile Storage is the storage that stays no mater the state of a server of a computer, powered or not. It is also referred as persistent or long-term storage. A good example of a Non-Volatile storage is hard drive in your computer or server and the flash USB Stick that we all carry around now. CD ROM and Magnetic Tapes are also qualified as Non-Volatile storage even though they are less frequently used these days.

The different type of storage that we mentioned earlier here are picked and choose during the design of a storage architecture due to their speed. Volatile Storages are usually faster than Non-Volatile memories. The diagram bellow illustrates very well that concept. Items on the top of the pyramid are faster than that bellow. On the top of the pyramid seats the Volatile Storage and below are the Non-Volatile.

Now that we have covered some of the basic knowledge of Storages, let’s move to further and talk about storage in the cloud.

Non -Volatile

I will be focusing on Amazon Web Service or AWS here in this blog post and particularly on Block storage EBS (Elastic Block Store), Object Storage S3(Simple Storage Service) and File Storage EFS (Elastic File Store).

Amazon via AWS offers block storage on the cloud also and it is called EBS (Elastic Block Store). An EBS (Elastic Block Store) is an easy to use, scalable and high-performance and low latency block storage service that provide persistent block level storage for Amazon EC2 instances.

SSD Backed volume types are intended for workloads that require low latency in need to High IOPS while HDD Volumes are intended for workload with demand for high throughput.
When dealing with EBS Volumes, customers have the choice to pick and choose between several types of block storages to meet the needs of various types of workloads.

SSD Backed Volumes are best suited for transactional workload like enterprise applications, relational and NoSQL databases, virtual desktops or boot volumes
HDD Backed Volumes on the other hand are best for throughput intensive workloads such as Bigdata applications, log processing, data warehouse applications. To learn more about EBS, check my other blog post here.

Object Storage S3(Simple Storage Service)

S3 or Simple Storage Service is the amazon branded version of Object storage. It is relatively simple and highly available (99.99% available or only down 50 minutes of downtown a year) and it is has a flat hierarchy and all the objects are at the same level and they accessible over an API Call.
S3 also some amazing durability with 11 nines. That translates to only 45 minutes to 1 hour year where you can’t retrieve your Data when you needed but will be there to be retrieved later.
S3 suited for the following use cases such as Backup and Archival, Static Website hosting, Content Delivery System, Disaster Recovery planning and big data analytics.

S3 also have multiple storage classed that allow data to be stored according to the frequency of its usage. Data in S3 can be store in one storage class and them move to other classes either manually or automatically via a process called lifecycle management. I will be covering Object Storage in more details in a later post here.

EFS/FSx or Elastic Block Storage

EFS or Amazon Elastic File System is the amazon version of file storage compatible only with Linux AMI with EFS, data is stored in a path-based system where data files are organized in folders and subfolders. EFS is a pay per use storage solution unlike EBS (No need to provision) and is best suited for applications that require scalable storage As Network attached file system, EFS can be attached to several EC2 instances in different AZ
EFS is mostly used in case can like file storage for Content Management systems, lift and shift operations and for application developments.

Volatile

Instance Store
Instance Store is the Volatile storage version of AWS storage. It is a non-persistent storage disk attached physically to an instance. Instance store are extremely fast due to the lack of latency but are wiped away when the instance is rebooted