Best Practices for Optimizing Amazon EBS Volumes

Amazon EBS (Elastic Block Store) is a storage service provided by Amazon Web Services (AWS). Each storage device in EBS is called a volume. AWS customers need to be familiar with Amazon EBS volumes to have their data stored on AWS cloud in the most efficient way. In this article, I'll explain what Amazon EBS volumes are, and how to optimize them for cost and performance.

What is an EBS volume?
An Amazon EBS volume is a block-level storage device that is designed to be used with Amazon EC2 instances. After a volume is attached to an instance, you can use it like a regular physical hard drive. You can format it with a specific file system, host operating system and applications, and have snapshots made from them.

Benefits of Using EBS Volumes

EBS volumes provide the following benefits:

• Data availability—every Amazon EBS volume is automatically replicated to other storage devices to prevent data loss and maintain high availability.
• Data persistence—an Amazon EBS volume exists independently from the instance it is connected to. You continue to pay for the volume usage as long as the data persists. EBS volumes that are attached to a running instance can automatically detach from the instance with their data intact when the instance is terminated, and the volume can be reattached to a new instance. This enables you to process and store the data on your volume indefinitely while using the processing and storage resources only when required.
• Data encryption—you can create encrypted EBS volumes with the Amazon EBS encryption feature. All EBS volume types support encryption. You can use encrypted EBS volumes to meet a wide range of data-at-rest encryption requirements. The encryption occurs on the server that hosts the EC2 instance, providing encryption of data-in-transit from the EC2 instance to Amazon EBS storage.
• Snapshots—Amazon EBS provides the ability to create snapshots of any EBS volume and write a copy of the data. The volume does not need to be attached to a running instance in order to take a snapshot. As you continue to write data to a volume, you can periodically create a snapshot of the volume to use as a baseline for new volumes. Snapshots of encrypted EBS volumes are automatically encrypted.
• Flexibility—EBS volumes support live configuration changes while being used. You can modify volume type, volume size, and IOPS (I/O per second) capacity without service interruption. ## Best Practices for Optimizing Amazon EBS Volumes Amazon EBS can be very expensive, depending on the volume type that you choose to use. There are four criteria that you should consider to optimize Amazon EBS volumes cost and performance:
• Disk space
• Disk latency
• Peak requests rate for I/O per second (IOPS)
• Maximum throughput

The following best practices will help you manage the costs of Amazon EBS volumes, without sacrificing performance:

1. Understand the costs associated with EBS volumes
In order to optimize EBS volumes, you should familiarize yourself with their various sizes and types.

• General Purpose SSD (gp2)━the general purpose volume type. Higher cost SSD storage with low latency.
• Provisioned IOPS SSD (io1)━low latency volumes and provisioned I/O performance. This is the highest cost SSD storage.
• Throughput Optimized HDD (st1)━lower cost magnetic volumes optimized for large amounts of sequential data transfer.
• Cold HDD (sc1)━lowest cost magnetic volumes with higher latency.
• Magnetic━high latency, and lower I/O performance.

You can save costs by matching the volume type to the maximum load it will receive and the required maximum latency. By default, AWS provisions a very fast high-cost volume. Many I/O needs can be met with lower cost magnetic drives. There are large savings to gain by moving less critical volumes to magnetic storage. Expensive io1 volume can be replaced by high performing gp2 volumes, sized to meet the actual I/O required by the application.

2. Assess PIOPS usage
Over-provisioned PIOPS (Provisioned IOPS) leads to unnecessary costs. Sometimes it is not clear where maximum read or write operations never approach the PIOPS. To get the best measure of the I/O for an io1 volume, use DiskReadOps and DiskWriteOps metric. This is the most conservative metric AWS has for io1 performance. Consumed Read/Write Ops combines reads and writes, and compensates for larger reads and writes. Using Consumed Read/Write Ops you can get a good idea of the actual need for I/O, and select the appropriate volume type.

3. Find candidates for migrating to magnetic volumes
Sc1 and st1 are much less expensive than gp2. They perform well in many applications which are not I/O intense, or do not require latency of less than one millisecond. Volumes with high throughput are good candidates for st1. To evaluate a transition from gp2 to sc1 or st1, it is recommended to run volume metric reports over significant time to estimate the average I/O levels.

4. Terminate detached and idle volumes
AWS charges for volumes at a GB/hour rate, whether they are attached to your Amazon cloud service or not, and regardless of whether they were accessed. By monitoring for unattached and idle volumes you can avoid paying for unnecessary drives.

Wrap Up

Optimizing EBS Volumes can result in significant cost savings. High-cost io1 volumes can often be migrated to gp2. Gp2 volumes can often be migrated to magnetic storage with significant cost savings. Amazon EBS Volumes user should be aware of the options to choose the best volume which is most suitable for the budget and required performance.