AWS Storage Options

When I was nine I discovered QBASIC on my parents' DOS computer. I got addicted immediately but, in the nineties, we didn't have stack overflow where I could just copy-paste example code.

What I did have were a few books that I found at a thrift store with example programs printed in them. I would spend hours copying the code (especially since I used the hunt-and-peck typing technique).

Once I had all of that code copied, I had no questions about where to store it. On my personal, 750 kb floppy disk.

Now I'm a cloud developer, and when I want to store files it's not as simple of a decision. There are all types of storage in AWS with a different use case for each. In this article, I'm going to walk through the most commonly used storage services and describe the common use cases for them.

Types of Cloud Storage

There are three different classifications of data stores: Object, File, and Block.

Object Storage

Object storage is the most cloudy type of storage. Objects, or files, are stored using a key and the contents of the store can be spread over multiple servers. This enables high availability and durability.

File Storage

If you have a file server, or a shared folder, on your network then you're already familiar with file stores. They store files in a folder structure and you can access them through a network.

Block Storage

This type of storage is all about performance. Essentially it just acts like a hard disk connected to whatever is using it.

What are the main services?

First things first, I'll briefly describe the services before getting into use cases.

S3 (Object)

S3 is one of the first services created on AWS. It's kinda the default choice for storage in AWS. It's fast, inexpensive, and can store a LOT of data. Unlimited storage space in fact. You can also store objects (files) up to 5 Tb in size!

S3 Glacier (Object)

While S3 is fast to access, S3 Glacier is very slow to get your files out of. As the name kind of implies. Getting files (archives) out of an S3 Glacier Vault can take between a minute and twelve hours depending on how much you want to pay for it.

Instance Store (Block)

Instance store is ephemeral storage just for your EC2 instances. Anything stored in an instance store gets destroyed when the EC2 instance that it's attached to is terminated. You can also only access an instance store from a single instance.

Elastic Block Store (Block)

Elastic Block Store is another storage solution for EC2, but unlike an instance store, its data is persisted when you terminate an instance. Also, you can share an EBS between multiple instances.

Amazon EFS (File)

Elastic File Store is a service that allows you to store files in S3 as if they were actually on your local network. You attach a VM to your network and it acts as a file server that stores the files in AWS.

How can you use them?

Of course, there's a reason that all of these different storage services exist. They all have different use cases. Let's look at a couple.

Long Term Storage

How do you save data that won't be accessed regularly? Maybe something that you're required to keep for a few years in case of an audit? That's the exact use case that S3 Glacier was designed for.

EC2 Server Caching

If you're running an EC2 instance that needs to cache some data, the instance store is what you're looking for. Since its content lives and dies with the EC2 instance that it's attached to, it's ideal for caches and buffers.

Apache Web Server Content

When you have a bunch of instances that all need to have the same data, an EBS may be the way to go. You could store everything in s3 and sync it as part of your bootstrap script, but using an EBS will ensure everything stays synced between server instances.

A File Server on Your Network

This one is probably obvious if you read my descriptions of the different services. EFS is designed to give you the convenience of a local file store, with the durability of cloud storage.

Conclusion

This article provided an introduction to the main storage services in AWS. You have seen the different classifications of storage, and AWS's main services for each type. You also saw a handful of use cases and the best service to use for each.