AWS Cost Optimization: Step by Step

DevOps and software engineering teams are increasingly reliant on public cloud infrastructure. Amazon Web Services (AWS) is the world’s largest cloud provider, and takes up a meaty share of the IT budget for many organizations.

As the excitement over cloud computing dies down, and the cloud becomes a normal part of daily operations, more teams are becoming interested in cost control and optimization—how to ensure they don’t waste resources on the cloud, and make optimal use of cloud pricing options, to ensure they conserve budget for other important projects and activities.

Cost control has additional operational benefits. By avoiding cloud sprawl and reducing the number of deployed resources to the necessary minimum, you reduce administration and management overhead, and improve security by reducing the threat surface of your cloud environment.

What is AWS Cost Optimization?

Moving to the cloud has changed the way organizations manage IT costs. The advent of pay-as-you-go, distributed resource provisioning, and unprecedented new managed service offerings, has forced organizations to change their approach to IT-based financial processes.

With AWS, you can build modern, scalable applications that make it possible to reduce costs compared to on-premise deployment, while continuously optimizing cloud spend. However, optimizing cloud costs is complex and in the absence of proper controls, AWS costs can quickly spiral out of control. AWS provides extensive resource managed options and multiple purchasing models you can use to design a cost-effective infrastructure.
AWS Cost Optimization: Best Practices for Managing Usage on AWS

Use the AWS Pricing Calculator

Image Source: AWS

The AWS Pricing Calculator is a web-based planning tool you can use to generate estimates for AWS use cases. You can use it to model a solution, research AWS service charges, and see the calculations behind a quote before building a solution. You can use it to plan your spending on Amazon Web Services, identify cost savings opportunities, and make informed decisions.

The AWS Pricing Calculator is useful for those who have never used AWS. It is also useful if you want to restructure or scale your AWS usage. No cloud or AWS experience is required to use the tool.

You can use the AWS Pricing Calculator to:

• Make AWS prices more transparent—view the calculations behind the estimated price of your expected service configuration. You can analyze the cost of your architecture by viewing quotes by service or service group.

• Use groups to organize your estimates—break up quotes into groups for a clearer service cost analysis.

• Share estimates—you can save a link to each quote to share with other stakeholders or view them. Quotes are stored on AWS public servers.

• Export your estimates—export quotes in CSV or PDF format.

Identify Amazon EC2 Instances With Low Utilization
AWS Cost Explorer is a tool that lets you receive automated cost optimization recommendations for Amazon resources. The tool’s Resource Optimization report gives you reports on idle or underutilized EC2 instances. You can either:

• Manually stop or scale these instances down to reduce costs.

• Automatically stop instances using AWS Instance Scheduler.

• Automatically resize EC2 instances using AWS Operations Conductor (based on Cost Explorer recommendations).

You can also use AWS Compute Optimizer to get recommendations for non-scaled instance types in your instance family. It provides recommendations for scaling within or across instance families, scaling up to remove performance bottlenecks, and recommends how to optimize EC2 instances that are part of an Auto Scaling group.

Note that unused instances can also have important implications on cloud security and compliance. Unused EC2 instances might contain sensitive data or systems, which external attackers or insider threats can compromise, resulting in damaging data breaches.

Delete Unattached EBS Volumes
When you launch an EC2 instance with attached storage, an Elastic Block Storage (EBS) volume is created. When an EC2 instance is terminated, in many cases the EBS volume remains (unless the Delete on termination checkbox was selected). This means orphaned EBS volumes continue to run, incur costs, and contribute to your monthly AWS bill.

In a large organization, there could be thousands of unattached EBS volumes, with multiple instances for the same data, depending on the lifecycle and uptime of the workloads. You can use AWS Cost Explorer to identify these unused EBS volumes and delete them.

Analyze Amazon S3 Usage

Image Source: AWS

You can use S3 analytics to analyze your storage access patterns over the past 30 days—this will give you a basic idea of your S3 usage and ideas on how to optimize it. Here are some suggestions for saving money by leveraging S3 Infrequent Access (S3 IA) or other archive tiers:

• Manually move datasets that are not frequently accessed to S3 IA, S3 Glacier, or S3 Glacier Deep Archive.

• Use lifecycle policies to automatically move these objects to lower-cost storage tiers.

• Use S3 Smart Hierarchy to automatically analyze objects and move them to the most appropriate storage tier.

Use Amazon EC2 Spot Instances to Reduce EC2 Costs
If your workload is fault-tolerant, you can save up to 90% of compute costs by using Spot Instances. Keep in mind that Spot Instances can be interrupted with notice of two minutes. Some examples of suitable workloads include big data, containerized workloads, CI/CD pipelines, web servers, high performance computing (HPC), and test and development servers.

EC2 Auto Scaling lets you launch regular on-demand and Spot Instances together, ensuring you have some instances that cannot be interrupted for critical management components. Auto Scaling automatically handles your Spot Instance requests and attempts to maintain your target capacity, even if Spot Instance go down.

Review and Modify EC2 Auto Scaling Groups Configuration
EC2 Auto Scaling Groups allow you to scale an EC2 instance fleet up or down as needed. Check your scaling activity by using the describe-scaling-activity CLI command or via the EC2 console. Analyze the results to see if you can adjust the scaling strategy, to avoid adding instances too aggressively. Also check your settings to see if you can reduce the minimum fleet and still meet the needs of your end users, which will reduce costs.

Evaluate Underutilized RIs
The main goal of Reserved Instances (RIs) is to commit to capacity in advance in order to receive discounts. So planning is essential to maximize value for money. Identify RIs that are not fully utilized and do one of the following to conserve costs:

• Reuse the RIs elsewhere, or redeploy them to an entirely new project or application, to avoid wasting the committed cost.

• Reuse existing workloads that use expensive on-demand instances to available RIs.

• If you don’t have any use for your RIs, you can sell them on the AWS RIs Marketplace.

Identify and Delete Orphaned Snapshots
When you terminate an EC2 instance, even when all attached EBS volumes are also deleted, it is easy to forget that the snapshots remain in S3 and the monthly fee continues to accrue.

This fee may also be higher than you think. Backups are mostly incremental, but the first snapshot is for the entire volume. Also, regular subsequent snapshots with a longer retention period may require the same amount of space as the first snapshot taken.

Even though snapshots are stored in S3, which has lower cost than general-purpose EBS volumes, you should consider deleting the original EBS snapshot associated with your volume. Over time, depending on the number of volumes and snapshots created, this can significantly reduce costs.

Conclusion

In this article, I explained the basics of AWS cost optimization and suggested an 8-step plan to optimize your cloud costs:

Use the AWS Pricing Calculator to get a basic idea of costs for new services and planned expansions.

1. Identify Amazon EC2 Instances with low utilization and use them or remove them.
2. Delete unattached EBS volumes, the “silent killer” of AWS costs.
3. Analyze Amazon S3 usage and either manually or automatically move data to a more optimal store tier.
4. Use Amazon EC2 Spot Instances to reduce compute costs by up to 90%.
5. Review and modify EC2 Auto Scaling Groups configuration to ensure you are scaling down as far as possible and not scaling up too aggressively.
6. Evaluate underutilized RIs and either use them or sell them on the AWS marketplace.
7. Identify and delete orphaned EBS snapshots which can add up to a huge expense.

I hope this is useful as you improve the cost-consciousness of your AWS cloud operation.