Amazon is the most widely used cloud provider and with new services being released regularly that seems unlikely to change any time soon. This makes sense—if you’ve already adopted one of AWS’ offerings, keeping your cloud services uniform can be easier to manage. EFS is just one of the offerings from AWS which is appealing because of its potential both as a stand-alone product and it’s usefulness as an integrated service.
An Intro to AWS EFS
AWS Elastic File System (EFS) is an automatically scaling system for Linux-based workloads that can be used with AWS or with on-premise resources. EFS is a fully managed service with a standard file system interface that offers file sharing through instances, AWS Direct Connect, or AWS VPN. It is designed to provide parallel access to thousands of Elastic Cloud Compute (EC2) instances for high aggregate throughput and I/O Operations per Second (IOPS) with low latency.
EFS is accessible across Availability Zones, Regions, and VPCs. Its primary uses are for lift-and-shift of enterprise applications, big data analytics, web serving, content management, media and entertainment workflows, and database backups.
How to Test Drive EFS
If you haven’t yet decided on AWS EFS, the best way to test it is by getting started with the free tier, which lets you use 5 GB of storage for 12 months. Doing so can help you get a feel for how the service works, what its limitations are, and develop some advanced file system techniques that are needed for successful operation.
To get set up you’ll need to create and launch an EC2 instance, create your EFS file system, connect to your instance, and mount the file system. This can be done either through the Management Console or the Command Line Interface (CLI) and a walkthrough can be found on AWS.
Once you’re set up, you can start testing its capabilities by writing files using single threads, multiple threads, with variable numbers of instances, or whatever other settings you think you might be interested in using.
Performance Tuning
Once you get EFS up and running in production, you’ll likely need to spend a bit of time tweaking your configuration to get the best performance. The following tips can get you started.
Increase Your Limits
In EFS, because instances are not used, I/O limits are dictated by storage type. You can select between General Performance, for low-latency and high scalability, or Max I/O, for large transfer volume but higher latency.
Whichever you choose, your new volumes will start with only .5MB/s baseline throughput and 7.2 minutes of 100MB/s burst throughput which can only be increased by increasing your file system size. To get around this, the simplest solution is to write dummy data to your system. Doing so will force both your baseline throughput and your burst limits.
Use Asynchronous Write
You can buffer pending write operations onto your EC2 instances if you enable the asynchronous write feature. Doing so will eliminate the need for a round trip from EFS to the client for each write and will reduce latency.
Learn From Metrics
AWS CloudWatch automatically collects metrics data for you, which can be accessed through CLI, API, or the console. This data can keep you updated on how your system is performing and provide you the information you need to troubleshoot any issues that arise. All of the metrics collected can provide value but burst credits, in particular, is one to keep an eye on as EFS users are frequently caught off-guard by them running out.
Separate Your Operations
To maximize your EFS performance, consider separating your latency-sensitive operations into individual volumes. This will grant them separate burst credits and throughput caps, thus protecting them from time-insensitive operations. You may not see the same performance as you would if your files were locally mounted but will get more consistent performance.
Don’t Run App Code
EFS is not able to provide the fast speed or large read volumes that deploying applications or managing codebases require. Instead, it is meant to be used as massively shared storage, used for media assets, asynchronous logs, or exported data files. It can certainly be used to store application data or even be accessed by applications but for actual deployment, you should stick to local file systems or containers.
Automate Backups
Unlike some of AWS’ other services, EFS does not come with a snapshot feature and you must use other methods for backing up your data. AWS Backup, custom scripting with Lambda, or via third-party tools, are all viable options. Backup has the added benefit of being able to manage the backup of your other AWS services while Lambda and custom scripting allow you the greatest flexibility.
Take Care With Mount Options
EFS default mount options are sufficient for most users so you should avoid adjusting them unless you’re sure that doing so will improve your system. During configuration, try to use Network File System (NFS) version 4.1 if possible to avoid missing out on protocol improvements.
One of the biggest benefits of EFS is its ability to attach to thousands of EC2 instances concurrently so make sure you’re taking advantage of this ability for increased aggregate throughput.
Wrap Up
EFS is just one of several services offered by AWS for file storage and it may be the right option for you. If you need to share files broadly and on a huge scale, it might even be your best option.
If you’ve been considering this service, I encourage you to play around with the free tier, keeping in mind the potential performance issues mentioned here. If you’ve already adopted this service, hopefully, this article gave you some useful tips for improving your system performance.
Latest comments (0)