Introduction:
By now we understand very well that:
Cloud == Pay for what you use
One of the biggest factor that attracts users to the cloud, is the ability to provision resources within minutes. You are able to setup a virtual server in the Cloud, with just a few clicks, in a matter of minutes.
Use Case:
Your organization's development team works from 08:00AM to 05:00PM Monday to Friday. The development team's Cloud Virtual Machines (Amazon EC2 instances) are running idle in their out of office hours. It is your responsibility to ensure that the dev environment stops at 05:30PM everyday, and resumes the next morning. The environment should remain down on the weekends. Automate this task.
Services :
Amazon Eventbridge (formerly called Amazon CloudWatch Events):
EventBridge is a serverless service that uses events to connect application components together, making it easier for you to build scalable event-driven applications. Event-driven architecture is a style of building loosely-coupled software systems that work together by emitting and responding to events. Event-driven architecture can help you boost agility and build reliable, scalable applications.
AWS Lambda:
AWS Lambda is a serverless, event-driven compute service that lets you run code for virtually any type of application or backend service without provisioning or managing servers. You can trigger Lambda from over 200 AWS services and software as a service (SaaS) applications, and only pay for what you use.
AWS IAM:
AWS Identity and Access Management (IAM), allows us to specify who or what can access services and resources in AWS, centrally manage fine-grained permissions, and analyze access to refine permissions across AWS.
Amazon EC2:
Amazon Elastic Compute Cloud (Amazon EC2) is a web service that provides secure, resizable compute capacity in the cloud.
Goal:
Setup a scheduler using the Amazon Eventbridge service which will trigger a Lambda function.
Lambda function will temporarily assume the defined IAM role. This IAM role will have an IAM policy associated with it. The IAM Policy is what grants the Role permissions to access our EC2 instances.
The Lambda function will stop/start our Amazon EC2 instances as per the schedule we define in Eventbridge.
This is part-1 of the blog, where we will :
i. Create the IAM Policy and associate it with an IAM Role.
ii. Create our Lambda functions: one to stop the EC2 instances and another to start the EC2 instance.
iii. Test the Lambda functions by triggering them manually, to check if they have the desired effect on our EC2 instances
Part-2 of the blog will tie everything together, where we will be configuring the Amazon Eventbridge rules and defining the Cron entry that will trigger our Lambda functions to stop/start our EC2 instances.
Pre-requisites :
- AWS IAM account (do not use root account) having:
- admin privileges
- access to AWS Management Console
- Understanding of AWS Identity based policies In case you are not familiar with IAM Policies or need a quick refresher, please refer my blog
- Atleast one EC2 instance created in the account (this is required for testing purpose)
Cost :
- None (if you have an AWS free-tier eligible account)
Implementation:
Let's begin! Login to the AWS Management Console as IAM admin user
1. Create IAM policy and IAM role
1.1 From the AWS Management Console navigate to IAM
1.2 From the left navigation panel, select Policies
and then click the Create Policy
button
1.3 Select JSON
1.4 Paste the below policy into the policy editor (delete all the pre-filled content) and click Next
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"logs:CreateLogGroup",
"logs:CreateLogStream",
"logs:PutLogEvents"
],
"Resource": "arn:aws:logs:*:*:*",
"Condition": {
"StringEquals": {
"aws:RequestedRegion": "eu-central-1"
}
}
},
{
"Effect": "Allow",
"Action": [
"ec2:DescribeInstances",
"ec2:DescribeRegions",
"ec2:StartInstances",
"ec2:StopInstances"
],
"Resource": "*",
"Condition": {
"StringEquals": {
"aws:RequestedRegion": "eu-central-1"
}
}
}
]
}
1.5 Give the policy a meaningful name and description. Review the permissions and then click Click Policy
1.6 From the left navigation panel, select Roles
and then click the Create Role
button
1.7 Select AWS Service
as the Trusted entity type
1.8 From the Service or use case
dropdown list, select Lambda
1.9 Click Next
to move to the next screen
1.10 From the policy dropdown list, select the policy we created in the previous step and click Next
1.11 Give the role a name and description
1.12 Verify the trusted entity (should be Lambda) and the associated customer managed IAM Policy. Click Create Role
Note the maximum duration this role can be assumed. Since we did not specify a duration, the default duration of 1 hour was assigned
2. Create Lambda Function to stop EC2 instances
2.1 From the AWS Management Console navigate to Lambda
Make sure you are in the desired AWS region. Change the region in case not.
2.2 Click Create Function
2.3 Select Author from scratch
2.4 Give the function a name
and select the desired runtime
.
In this demo we will be using
Python 3.11
as the runtime.
2.5 In the Permissions
tab, change the default execution role to Use an existing role
.
From the dropdown list, select the lambda assume role we created in the previous step.
2.6 Click Create Function
Good job keeping up so far! You are making good progress!
2.7 Scroll down a bit to reach the Code
section. You will notice a pre-filled Code source
section.
2.8 Delete the pre-filled contents and paste the below code in its place:
import boto3
region = 'eu-central-1'
def lambda_handler(event, context):
ec2_resource = boto3.resource("ec2", region_name=region)
# Apply filter to check for EC2 instances in 'running' state
instances = ec2_resource.instances.filter(Filters=[{'Name': 'instance-state-name', 'Values': ['running']}])
print("EC2 region is: ", region)
print("instances= ", instances)
# Count running instances
count = len(list(instances))
print(f"{count} instances running")
# Stopping EC2 instances
for instance in instances:
instance.stop()
print(instance, ": Stopping!!! .")
2.9 Click Deploy
2.10 In the Configuration
tab, select General configuration
and click Edit
2.11 Change the Timeout
value to 10 seconds
and click Save
Note: The timeout value may differ for each use case, so choose one according to your requirement
2.12 Click Test
to test if our lambda function is working as expected
Configure the test event.
Just an event name
will suffice. Our Lambda does not require any user input.
Click Save
Click Test
to execute the newly created Lambda function.
Review the output logged by the stop EC2 Lambda.
Important: For this to work you need to have atleast one EC2 instance running in eu-central-1 region
- Two EC2 instances were found to be in
running
state in the region defined within our Lambda functioneu-central-1
- Our Lambda attempts stopping each of these one-by-one (as coded in our
for
loop)
Notice the time duration of our Lambda execution - it is roughly 4 seconds. If we left the timeout value as default 3 seconds, our Lambda function would most likely have timed-out before completing the execution.
2.13 Go to the EC2 dashboard in the AWS Management Console (in the same region defined in your Lambda code) and check if the instances really stopped.
It worked!
3. Create the Start EC2 instance Lambda Function
3.1 Repeat steps to create another Lambda function and this time use the below Python code, which will start our EC2 instances.
import boto3
region = 'eu-central-1'
def lambda_handler(event, context):
ec2_resource = boto3.resource("ec2", region_name=region)
# Apply filter to check for EC2 instances in 'stopped' state
instances = ec2_resource.instances.filter(Filters=[{'Name': 'instance-state-name', 'Values': ['stopped']}])
print("EC2 region is: ", region)
print("instances= ", instances)
# Count running instances
count = len(list(instances))
print(f"{count} instances in stopped state")
# Starting EC2 instances
for instance in instances:
instance.start()
print(instance, ": Starting!!! .")
3.2 Test the Lambda
Review the execution result of the Lambda function.
- 2 EC2 instances were found in
running
state in the defined regioneu-central-1
- Both are stopped one after the other, with their respective EC2 instance IDs logged.
- The execution took ~4 seconds.
3.3 Verify if the EC2 instances started in the EC2 dashboard
Pheewww!!
That was a lot of hand's on work!
I hope this gave you good exposure on how to navigate the AWS Management Console and helped grow your understanding on how multiple services can be used together to build a simple solution !
Conclusion:
In this blog we discussed our use case and associated architecture. We created an IAM execution role which allows our Lambda functions to access our EC2 instances. We even created the EC2 stop and EC2 start Lambda functions and tested them in isolation.
Stay tuned for the part-2 of this blog, where we complete our architecture by configuring Eventbridge to trigger our Lambdas.
Bonus:
Now that you have understood how to create the Lambda functions, feel free to play around with these and modify as per your desired use case. In our demo we restricted the EC2 task to a single region, this can easily be extended to include multiple AWS regions.
You can even enhance the instance filters on the available EC2 instance states!
Additionally, you make the code more dynamic, by filtering the EC2 instances based on defined Tags
!
Filter based on instance state and defined tags:
instances = ec2_resource.instances.filter(Filters=[{'Name': 'instance-state-name', 'Values': ['running']},{'Name': 'tag:environment','Values':['qa']}])
The valid values EC2 instance states are:
- pending
- running
- shutting-down
- terminated
- stopping
- stopped
EC2 instance life cycle:
Top comments (2)
please check it out very similar implementation..
dev.to/awsmantra/eventbridge-sched...
Nice article. Detailed description along with images make it more easy to understand