DEV Community

Gabriela Menocal for AWS Community Builders

Posted on • Edited on

How to choose the appropriate storage type in AWS ?

Choosing an appropriate storage is one of the most important decisions while either creating an app, migration or simply creating something completely new and unique.

Nowadays, we have multiple options of storage types to choose from, but when deciding you will need to consider the purpose for it. For example, will it be for mobile applications, archiving, sharing content, who is going to use the application and so on…

In this article I am not hoping to give the answer to all doubts related to this, but share a little bit of my experience on the field and share some tips that have helped me when I need to design about it.

First let’s briefly describe the most common file storages types and some of their main features of each of them:

File Storage Block Storage Object Storage
No Metadata Limited Metadata Metadata
High IO performance Hierarchical System Unique identifier
Write/Read efficiency Limit in Scalability and performance. Better for Read only applications and Distribution
Greater Storage efficiency Protocols: NFS/CIFS, SMB, etc. Protocols: HTTP, HTTPS, REST operation (GET, PUT, DELETE, etc.)

And some other factor in data management should be identified for our type of data: Structured or Unstructured data. The majority of the data we use and access everyday is normally unstructured, such as videos, images, emails, spreadsheets, word processing files, etc.

This is a big and deep topic, however this is out of the scope of this blog, but because I know there are people like me who would like to go further on this, I found these articles, which I believe are a great start:

So, going back to our main topic, as a rule of thumb, if the data can be displayed in a row, columns and/or SQL DB without or minimal modification, it means that most likely you are working with structured data.

Please do not forget that normally unstructured data requires more storage than structured data, therefore choosing the right storage is crucial to keep a good performance for the apps and services.

Now that we've discussed different data types, the next question arises, how can we use this knowledge to choose the right storage when using AWS?

In order to connect the two points, I would like to describe a little bit of the storage types offered by AWS. If we go to the AWS official documentation page we will notice that they offered the following (by the time this blog was written):

AWS Storage types

I highly recommend go to the official documentation to understand a little bit more each of these storages types, or read some of these blogs written by other members of AWS Community Builders, for example check these:

There are many more article explaining to this and different types of topics and services, please take a look in the channel
AWS Community Builders Channel.

So, I mapped the two points in the following table, which will give you a general idea of the type of data for which the storage is better suited for:

Storage Type Block Storage File Storage Object Storage Volume Based Storage Tape Storage
Amazon S3 😃
AWS Backup 😃 😃 😃 😃 😃
Amazon EBS 😃
Amazon EFS 😃
AWS Elastic Disaster Recovery NA NA NA NA NA
Amazon FSx 😃
Amazon S3 Glacier 😃
AWS Snow Family NA NA NA NA NA
AWS Storage Gateway 😃 😃 😃

Please use as a reference for table 1:

  • 😃 => This is the storage type.
  • ⛔ => It is not suitable for it.

And I also mapped the Storage features with AWS Storage offering, to give you a general idea and it can be used as a reference:

Disclaimer: Not everything is written in stone, so please make sure to use this tables as reference and general guidelines only.

Storage Type Metadata High IO Performance Distribution Scalability Data Modification
Amazon S3 😃🙌 😐 😃🙌 😃🙌 😃
AWS Backup 😃 😃 😃 😃 😃
Amazon EBS 😃🙌 😐 😃🙌
Amazon EFS 😐 😃 😃 😃 😃
AWS Elastic Disaster Recovery NA NA NA NA NA
Amazon FSx 😐 😃🙌 *(Lustre) * 😃 😃 😃
Amazon S3 Glacier 😃🙌 😃 😃🙌 😐
AWS Snow Family NA NA NA NA NA
AWS Storage Gateway 😃 😃 😃 😃 😃

For anyone who is curious about it, this is the Link for the Lustre Project: https://www.lustre.org/.

Please use as reference for table 2:

  • 😐 => Possible but with some caveats or there are some limitations or restrictions.
  • 😃 => It will work but there might be other options.
  • 🙌 => This is what it was built for.
  • ⛔ => it is not suitable for it.

I hope you find this post useful and please if you have any comments, do not hesitate in letting me know.

Top comments (0)