Automate EC2 Instance Shutdown with Auto Scaling Groups to Minimize AWS Costs

Shutting down EC2 instances when they are not in use is an effective way to manage cloud costs. In my previous post I described a method for starting and stopping EC2 instances on a schedule using CloudWatch Event Rules and Lambdas.

Launch Templates, in conjunction with Auto Scaling Groups (ASG), provide another solution which achieves the same result. This solution uses the ASG's ability to scale-out and scale-in on a schedule. More information about the use cases for each solution can be found in the previous post. This post focuses on setting up Launch Templates and ASGs. It assumes you have a general working knowledge of each of these AWS services.

Launch Templates and ASG

Employing this method to scale-out and scale-in EC2 instances on a schedule can be broken down into three steps:

1. Create a Launch Template from an EC2 instance

2. Create the Auto Scaling Group

3. Create schedules to scale the ASG

Step 1: Create a Launch Template from an EC2 instance

From the AWS documentation, a launch template

...specifies instance configuration information. It includes the ID of the Amazon Machine Image (AMI), the instance type, a key pair, security groups, and other parameters used to launch EC2 instances...

Launch Templates are used when scaling EC2 instances with ASG. Whenever an ASG needs to scale-out and create a new EC2 instance, the instance is configure using the settings from the launch template.

To create a launch template from an existing EC2 instance, navigate to the EC2 services console and click on the "Instances" link. Start the Launch Template wizard by selecting an existing instance and choosing Actions | Image and templates | Create template from instance menu options.

There are many settings in the wizard. I've listed the relevant settings by section below. Any settings not called out specifically can be left as the default.

After completing the form, click the "Create launch template" button to validate and save the template. A common source of invalid templates is the combination of selected security groups and subnets. If you have an invalid template, be sure to verify you have selected the proper values for those settings. Once the template is valid, you will see a success message similar to the one shown below.

This dialog presents three links. To start the wizard which guides the creation of the ASG, click on the "Create Auto Scaling group" link.

Step 2: Create the Auto Scaling Group

The AWS documentation describes an ASG as

...a collection of Amazon EC2 instances that are treated as a logical grouping for the purposes of automatic scaling and management...

ASGs provide functionality for scaling the number of running instances. It is this feature that makes ASGs ideal for this solution.

Like the launch template wizard, there are many settings in the ASG wizard. The ASG wizard walks through seven steps, four of which are optional. For brevity's sake, I focused on the values for the required steps and skipped over the optional ones. Any settings not called out specifically can be left as their default unless you have special needs for your unique configuration.

Step one asks for an ASG name and a Launch Template. Enter "schedule-instance-asg" as the name and select the "schedule-launch-template" Launch Template. Click the "Next" button to navigate to step two.

The first section in step two asks for the Instance Purchase options. Instance purchase options provide a way to specify how instances are to be distributed. Select the "Combine purchase options and instance types" option.

The settings in the Instance Distribution section allows further cost optimization by providing choices for how EC2 instances are paid. ASGs allow for two distribution options, On-demand and Spot. Spot instances are cheaper but have very specific use cases. You can learn more about EC2 pricing here.

Since this is a post about saving costs I decided to use spot instances instead of On-demand. You will have to decide what is best for your situation. To use spot instances provide the following values when filling out the form:

The Instance Type section allows for provisioning the type of VM in the ASG. The instance type value will be pre-selected with the value specified in the launch template that was created in step one. You can modify this setting or leave it as the default. A nice benefit that comes from the scheduling of instances to shut down is that you can choose a beefier VM knowing that you are saving money when the instance shuts down. After selecting the instance type, move onto the Network section.

In this section, select the VPC and subnets where the instances are to be launched. The default VPC is pre-selected so be sure to change it if you are using a non-default VPC. That's all for wizard step two. Click the "Next" button until you reach step five where you can add notifications for ASG events.

SNS Topics are used to send notifications whenever the instances in an ASG are started or stopped. To setup notifications, select an existing topic from the SNS Topic dropdown, or click the "Create a topic" button to create a new SNS topic. Provide one or more email addresses in the With these recipients textbox and select one or more of the following ASG events: Launch, Terminate, Fail to launch, Fail to terminate, as shown below.

Click the "Skip to review" button to jump to the final step in the ASG wizard. Here you can review the settings and configurations chosen in all of the previous steps. After verifying everything, click the "Create Auto Scaling group" button to create the ASG.

Step 3: Create schedules to scale the ASG

The final step in this process is to setup the scale-out and scale-in schedules. Select the "schedule-instance-asg" ASG from the list, click on the "Automatic Scaling" link and scroll down to the Scheduled actions section. Click "Create scheduled action" to open the form shown below.

Use this form to create the schedule that will scale-out, i.e. startup the instance(s). Provide the values from the table below to scale-out to one instance at 7AM Est, Monday through Friday.

Click the "Create" button to save the schedule. Repeat the process to create the schedule to scale-in, i.e. shutdown, the instance(s). Provide the following values to run the schedule at 8PM Est, Tuesday through Saturday.

Click the "Create" button to save the scale-in schedule.

Logging and Monitoring

One of the features of ASG is its inherent ability to log activity and monitor instances. You can view the logs by selecting the "schedule-instance-asg" ASG from the list and clicking the "Activity" link. The data shown includes the status and runtime of each ASG. It includes any deltas in the number of running instances among other useful information.

Instance monitoring is achieved by clicking on the "Monitoring" link. ASGs monitors a wealth of data including Minimum and Maximum Instance counts, the number of instances running and terminated, and resource usage of the EC2 instances in the ASG.

Conclusion

Controlling costs in the cloud can seem like a full time job (actually, it probably is). If left running, EC2 Instances quickly accrue fees and can lead to sticker shock when you get your monthly invoice. Automating the shutdown of EC2 Instances is one very effective method for controlling costs.

Part one of this series focused on using CloudWatch Event Rules and Lambdas to automate EC2 instance startup and shutdown. Here we looked at using Launch Templates and Auto Scaling Groups to achieve the same result. While each has its own pros and cons, both are extremely effective techniques for keeping costs under control. Drop me a note in the comments and let me know how you control costs in your cloud environment using these, or any other methods.