Balancing between Cost and Performance of EFS. A guide to drive the best dollar performance using Amazon EFS.

Introduction

Performance and cost optimization are two different extreme ends of a pole. Identifying your requirements and selecting the right performance characteristics are essential for an optimized cost solution/services.

AWS Elastic File System (EFS) is a shared file system offering from AWS works with Network File System (NFS) protocol. It is one of the shared file systems used in various architectures. EFS resemebles Network Attached File System (NFS) from traditional storage solutions. Capabilities of EFS has been diversified in last few years to become the befitting storage choice for Elastic Kubernetes Services (EKS) , Elastic Container Services (EKS), Fargate and Lambda functions besides mounting capabilities from AWS EC2 instances(with NFS Protocol).

For Implementation please refer;

1. Mount EFS in EKS:- https://aws.amazon.com/premiumsupport/knowledge-center/eks-pods-encryption-efs/
2. Mount EFS in ECS:- https://aws.amazon.com/premiumsupport/knowledge-center/ecs-fargate-mount-efs-containers-tasks/
3. Mount EFS in lambda:- https://aws.amazon.com/premiumsupport/knowledge-center/efs-mount-with-lambda-function/

Unique capabilities of EFS.

This shared file system has many distinctive forte which speeds the adoption as a File Storage.

1. Automatic Capacity management(elastic):- Zero storage capacity management is one of the greatest feature, it can grow from zero to Terabyte of capacity transparently . AWS allocates storage initially, monitor capacity demand and add/shrink capacity as it grows.
2. True elastic in nature:- AWS increase and resize/decrease the storage allocation based on files you add/remove. Storage consumption cost is based on the actual storage capacity used.
3. Multi Regional replication:- Out of the box configurations are available to replicate your EFS volume to other region or Azs within the region. Amazon EFS Replication is nearly continuous, designed to provide a recovery point objective (RPO) and a recovery time objective (RTO) of minutes. This makes as a storage of choice if multi regional recover is in picture.
4. Multi client support:- EFS shares can be accessed within the region, across regions and from Datacenter through Direct connect.

For details, refer : https://aws.amazon.com/efs/features/

EFS file system has two extends , performance(Throughput) and optimized cost(storage tiers). EFS service offer three options of throughput and few options for cost optimization.

Throughput modes

EFS provides 3 throughput modes for each file system.

Bursting (default):- This is the default throughput mode which scales throughput based on the storage capacity at of 50 KiBps per each GiB of storage. In this mode, Burst credits accrue when the file system consumes below its base throughput rate, and are deducted when throughput exceeds the base rate.

When burst credits are available, a file system can drive throughput up to 100 MiBps per TiB of storage, up to the Amazon EFS Region's limit, with a minimum of 100 MiBps. If no burst credits are available, a file system can drive up to 50 MiBps per TiB of storage, with a minimum of 1 MiBps.

To deep dive, refer:- https://docs.aws.amazon.com/efs/latest/ug/performance.html#bursting

Elastic:- In contrasts to the baseline and burst credits way of Bursting throughput mode, Elastic throughput mode of EFS automatically scales throughput performance up or down to meet the needs of your workload activity. This is suited for spiky workloads where the throughput cannot be predicted and change dynamically. Elastic Throughput can drive up to 3 GiBps for read operations and 1 GiBps for write operations per file system, in all AWS Regions.

Provisioned Throughput:- In this mode, you define the throughput requirement of your file system irrespective of the storage capacity needs, You pay throughput cost dedicately along with the storage cost.

Provisioned Throughput can drive up to 3 GiBps for read operations and 1 GiBps for write operations per file system, in all Regions.

Re-Configure throughput mode

The above mentioned throughput mode can be changed for a file system with certain restrictions. To dive the required performance on the file system , you need to switch these by evaluating the performance charteristics and cost factors.

Mode change restrictions

the following actions are restricted for a 24-hour period:

1. Switching from Provisioned Throughput mode to Bursting or Elastic Throughput mode.
   

2. Decreasing the provisioned throughput amount.
   

How to determine the right throughput mode

EFS reports its performance usage in AWS CloudWatch services. The Metrics relevant to identify the right choice of throughput is “PermittedThroughput”. It measures the amount of allowed throughput for the file system. This value is based on one of the following methods:

For file systems in Bursting Throughput mode, this value is a function of the file system size and BurstCreditBalance. If the metrics shows value consistently at or near zero, consider switching to Elastic Throughput or Provisioned Throughput mode to get additional throughput.

For file systems in Elastic Throughput mode, this value reflects the maximum write throughput of the file system.. You may also use “MeteredIOBytes “along with “PermittedThoughput”. When the values for MeteredIOBytes and PermittedThroughput are equal, your file system is consuming all available throughput. For file systems using Provisioned Throughput mode, you can provision additional throughput.

Similar to various throughput options, there are various storage classes available ; based on the access needs of the EFS files, they can be moved to Infrequent Access (IA) tiers through Life Cycle Management. An efficient selection of the right storage class gives you the best dollar performance. Let us have a close look on the tiers.

Data stored in the EFS standard or EFS One-Zone standard can move their files to EFS IA or One Zone IA through life cycle management. AWS:Reinvent 2022 has an announcement of “1”day transition policy added to the existing 7, 14, 30, 60, and 90 days. This is a simple configuration for data moved between different tiers , Standard to EFS IA . Data stored on the IA classes are 47 percent lower than the standard tiers.
EFS One Zone Storage is an optimal solution for non-critical workloads that can sustain from Availability Zone (AZ) failures. Typical use cases are non-production file system, re-producible files, temporary storage etc.

Amazon EFS Intelligent-Tiering, a new EFS lifecycle management feature that helps to move file between tiers based on the access pattern of the file. Intelligent-Tiering helps to move the file back to the standard tier transparently when files are accessed from IA class. This helps to reduce the retrieval charges of IA classes.

Conclusion
Identifying the right balance of Throughput and performance tiers of EFS storage classes is crucial to get the right dollar performance.