DEV Community

Cover image for Automate AWS Cost & Usage report using Event Bridge, Lambda, SES, S3 & AWS Cost Explorer API
Girish Bhatia
Girish Bhatia

Posted on • Updated on

Automate AWS Cost & Usage report using Event Bridge, Lambda, SES, S3 & AWS Cost Explorer API

Do you often wait for a report on cost usages for your cloud resources to get a view of overall cost?

AWS cost explorer API provides a way to automate the creation of cost & usage report that can be invoked on demand or can be scheduled using other AWS services so that a formatted report can be dropped in your inbox on a regular cadence (weekly, monthly etc.)

Let's see how to automate AWS cost an usage report using Event Bridge scheduler, Lambda and AWS Cost Explorer API.

I will create the cost & usage report using Lambda function, will share it via email to the verified email, will post it to a S3 bucket for record keeping or further processing by other services and will schedule this via event bridge scheduler.

In the realm of cloud computing, resource provisioning is effortlessly streamlined, offering invaluable agility to businesses for a myriad of use cases. This includes accelerating solution delivery, enhancing agility in infrastructure, computing, and scalable storage, as well as facilitating backup, integration, and more. Nevertheless, this convenience comes at a cost; each initiated resource contributes to the overall expense of cloud usage. Therefore, it is imperative to regularly scrutinize both costs and resource utilization, optimizing them in accordance with the specific needs of the business.

Let's review our use cases:

β€’ Generate a monthly cost and usage report for each AWS service.
β€’ Email the generated report to the assigned email recipient.
β€’ Post the generated report in JSON format to an S3 bucket.
β€’ Schedule this report using the EventBridge scheduler.
Enter fullscreen mode Exit fullscreen mode

I am generating my lambda function using AWS SAM, however similar can be created using AWS Console. I like to use AWS SAM wherever possible as it provides me the flexibility to test the function without first deploying to AWS cloud.

Here is the architecture diagram for our use case.

Image description

Let's see the steps to create this automated cost & usage report:

Create a S3 Bucket

Since our business use case requires storing the cost & usage report data in JSON format in S3 bucket, first we will need to create a S3 bucket. My s3 bucket is as below, but you can name the bucket as you wish.

Image description

Create a SAM template

I will create a SAM template for the lambda function that will contain the code to invoke cost explorer API for my account. Lambda function can be created without the SAM template however, I prefer to use Infra as Code approach since that allow for easy recreation of cloud resources. Here is the SAM template for the lambda function.

Image description

This SAM template is defining a Lambda function with timeout and few other parameters. I have some environment variables also defined as those variables provide flexibly of updating s3 bucket, region, report and account at runtime.

Create a Lambda Function

Lambda function is the core of this automated solution. This function will have the code for the business requirement of creating the cost & usage report for AWS cloud resources being consumed. This lambda function also has sub-functions to generate email report and uploading the JSON data in s3 bucket. Let's look at the code behind it.

Image description

Create Event Bridge Scheduler

Event Bridge schedule is another AWS service that I will use to complete the use case for automating the cost and usage report.

Image description

This scheduler can take the scheduling as CRON job pattern. I am creating a CRON job to generate the report at 12am on the first day of the month.

Image description

Last step is to integrate this schedule with the lambda function created in the earlier step. This integration can be performed using the AWS console.

Image description

Once this integration is completed, entire solution to generate automated cost & usage report is in place. Once schedule has invoked the function on a defined schedule, you will see the report posted in the S3 bucket. An example is shown below:

Image description

If you download and open this report, you will see the data in JSON format. An example as below:

Image description

A sub-function being invoked in the Lambda is using another AWS service to send this report via email to a pre-determined email.

Image description

"That's it! You now have a solution that provides an automated way of receiving the cost and usage report on a monthly basis via email. Additionally, a JSON format report is posted in an S3 bucket for storage and further processing. A schedule has been created using EventBridge, which will invoke this Lambda function on a monthly basis. This monthly notification of summarized costs by service meets the needs of our business use case and provides an automated method to generate and receive these reports!"

Thanks for reading!

Click here to get to YouTube video for this solution.

π’’π’Ύπ“‡π’Ύπ“ˆπ’½ ℬ𝒽𝒢𝓉𝒾𝒢
𝘈𝘞𝘚 𝘊𝘦𝘳𝘡π˜ͺ𝘧π˜ͺ𝘦π˜₯ 𝘚𝘰𝘭𝘢𝘡π˜ͺ𝘰𝘯 𝘈𝘳𝘀𝘩π˜ͺ𝘡𝘦𝘀𝘡 & π˜‹π˜¦π˜·π˜¦π˜­π˜°π˜±π˜¦π˜³ 𝘈𝘴𝘴𝘰𝘀π˜ͺ𝘒𝘡𝘦
𝘊𝘭𝘰𝘢π˜₯ π˜›π˜¦π˜€π˜©π˜―π˜°π˜­π˜°π˜¨π˜Ί 𝘌𝘯𝘡𝘩𝘢𝘴π˜ͺ𝘒𝘴𝘡

Top comments (0)