Usually when we should manage a large amount of AWS accounts, saving costs at scale is not a simple task, we need to use third-parties tools or code scripts to do that. In this post I will show you how to manage EC2 instances running time efficiently using Lambda and EventBridge Scheduler based on invented requirements.
Disclaimer
I describe an hypothetical state to walk through an use case and solution proposed.
Current Status
I am a member of the IT team that manages AWS resources for a company around the world. We have some EC2 instances that run software to be used for our company’s employees during work days in office hours.
Requirements
We need to establish a mechanism to save cost keeping in mind that employees are allocated in different time zones.
Solution proposal
We can purchase saving plans, or reserved instances, but that does not make sense because we don’t need 24x7 EC2 running, we only want these instances up from Monday to Friday in office hours. So the most efficient way, in this case, is to maintain instances running for the period of time that users need to access to the software installed and stop them the rest of the time.
How can we do that? I will show you an example with Lambda to start and stop instances and EventBridge Scheduler to manage Lambda invocations.
For this example I deploy some EC2 instances that are being used from Mexico and Spain users and all of them are deployed on eu-west-1 region (Ireland). To manage instances based on country I have tagged them with the “country” key and the value of this tag is based on ISO 3166-1 alpha-2 country code. That means that I should manage two time zones to support users in two different countries.
Below you can find a diagram with resources involved in the solution:
Step by step
1.- Launch EC2 Instances
I use the smallest instance type for this example based on a Linux distribution, you can launch instance following any of methods explained in Launch your instance topic from EC2 user guide. I have used two instances but you could launch more if you want.
The most important thing for this example is to tag all of the instances with key:country value:es or value:mx
2.- Deploy lambda functions for starting and stopping instances
In this case I have code two different Lambdas coded in python, one for starting instances and other for stopping instances. I hope you can find code self-explained.
I have decided to deploy one lambda per action and country, so there are 4 lambda functions.
To deploy lambda functions you can follow the steps described in AWS Lambda Developer Guide.
At the end of this section you can find code for each function and custom policies to be attached to Lambda's role.
Keep in mind that you need to create a Role to be attached to Lambda function and must contains a custom policy to grant permission for Lambda execution.
After deployment you should have 4 Lambda functions
Lambda code for starting instances
import logging
import boto3
logger = logging.getLogger()
logger.setLevel(logging.INFO)
ec2 = boto3.resource('ec2')
#Change value based on ISO 3166-1 alpha-2 country code
COUNTRY_CODE = 'es'
STOPPED_INSTANCE_CODE = 80
def is_stopped(instance):
if instance.state['Code'] == STOPPED_INSTANCE_CODE:
return True
return False
def has_country(country, instance):
tags = instance.tags
for tag in tags:
if (tag['Key'] == 'country') & (tag['Value'] == country):
return True
return False
def get_stopped_instances_id(country):
stop_instances_id = []
instances = list(ec2.instances.all())
logger.info("Number of instances found: {}".format(len(instances)))
for instance in instances:
if has_country(country,instance) & is_stopped(instance):
stop_instances_id.append(instance.instance_id)
return stop_instances_id
def start_instances(country):
stopped__instances_id = get_stopped_instances_id(country)
logger.info("{} number of instances that will be start".format(str(len(stopped__instances_id))))
for id in stopped__instances_id:
ec2.Instance(id).start()
logger.info('Instance with id {} has been started'.format(id) )
def lambda_handler(event, context):
country=COUNTRY_CODE
logger.info('Starting instances of: {}'.format(country))
start_instances(country)
Custom policy for Lambda execution's Role (start instances)
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": [
"ec2:DescribeInstances",
"ec2:StartInstances"
],
"Resource": "*"
},
{
"Sid": "VisualEditor1",
"Effect": "Allow",
"Action": [
"logs:CreateLogStream",
"logs:CreateLogGroup",
"logs:PutLogEvents"
],
"Resource": "arn:aws:logs:*:*:*"
}
]
}
Lambda code for stopping instances
import logging
import boto3
logger = logging.getLogger()
logger.setLevel(logging.INFO)
ec2 = boto3.resource('ec2')
#Change value based on ISO 3166-1 alpha-2 country code
COUNTRY_CODE = 'es'
RUNNING_INSTANCE_CODE = 16
def is_running(instance):
if instance.state['Code'] == RUNNING_INSTANCE_CODE:
return True
return False
def has_country(country, instance):
tags = instance.tags
for tag in tags:
if (tag['Key'] == 'country') & (tag['Value'] == country):
return True
return False
def get_running_instances_id(country):
running_instances_id = []
instances = list(ec2.instances.all())
logger.info("Number of instances found: {}".format(len(instances)))
for instance in instances:
if has_country(country,instance) & is_running(instance):
running_instances_id.append(instance.instance_id)
return running_instances_id
def stop_instances(country):
running_instances_id = get_running_instances_id(country)
logger.info("{} number of instances that will be stop".format(str(len(running_instances_id))))
for id in running_instances_id:
ec2.Instance(id).stop()
logger.info('Instance with id {} has been stopped'.format(id))
def lambda_handler(event, context):
country=COUNTRY_CODE
logger.info('Stopping instances of: {}'.format(country))
stop_instances(country)
Custom policy for Lambda execution's Role (stop instances)
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": [
"ec2:DescribeInstances",
"ec2:StopInstances"
],
"Resource": "*"
},
{
"Sid": "VisualEditor1",
"Effect": "Allow",
"Action": [
"logs:CreateLogStream",
"logs:CreateLogGroup",
"logs:PutLogEvents"
],
"Resource": "arn:aws:logs:*:*:*"
}
]
}
3.- Configure EventBridge scheduler
We have code for starting and stopping instances, now we're going to configure EventBridge escheduler to run these lambdas. For this example I only mantain EC2 instances running 2 hours a day
I have configure four different schedulers:
- Start instances for Spain from Monday to Friday at 9:00 AM (Europe Timezone)
- Stop instances for Spain from Monday to Friday at 11:00 AM (Europe Timezone)
- Start instances for Mexico from Monday to Friday at 9:00 AM (America/Mexico City Timezone)
- Stop instances for Mexico from Monday to Friday at 11:00 AM (America/Mexico City Timezone)
Below you can find a screenshot of 4 schedulers
Below you can find a scheduler configuration for starting instances:
You can learn more about how to manage Cron-based schedules here
Top comments (0)