Solving Cloud Cost Monitoring

In any cloud-based environment, keeping track of costs is crucial for managing budgets and avoiding unexpected expenses. However, AWS does not provide a built-in service specifically designed for automated cost reporting and notifications. This often leaves teams manually checking and reporting costs, which can be time-consuming and prone to oversight. To solve this problem, we developed a homemade solution to automate cost reporting, delivering daily updates via email and Microsoft Teams.

The Challenge: Manual Billing Checks
In today's fast-paced startup environment, companies rely on AWS services to quickly develop and scale their products. Imagine a small team of developers, full of innovative ideas, exploring AWS's extensive offerings. They make use of various services, integrating them smoothly into their projects. As the month goes on, the team focuses intently on building and improving their product, unaware of the accumulating costs. When the billing statement arrives, they are often shocked to find expenses far exceeding their expectations. This scenario isn't limited to startups; established companies running POCs on AWS frequently face similar surprises.

To help manage these costs, AWS provides a budgeting service that allows users to create customized budgets, set spending thresholds, and define total budgets. Notifications are sent via email when spending surpasses these thresholds, detailing Current Spend, Budgeted Price, Forecasted Spend, Current vs. Budgeted, and Forecasted vs. Budgeted amounts. However, this budgeting service lacks granularity in showing which specific services were used and their respective costs.

To bridge this gap, leveraging AWS Lambda functions and the AWS SDK Version 3 offers a more detailed breakdown of service usage and associated costs. This approach enhances visibility and control over AWS expenditures, enabling teams to manage their budgets more effectively and avoid unexpected expenses.

Solution
To ensure timely cost monitoring, we implemented an automated cost reporting system using AWS services. Here's how our solution works

1. AWS EventBridge: This service schedules a trigger every day at 7:00 AM.
   (Note: The Reason we set the trigger at 7:00 AM because of AWS billing information is not updated in real-time. there is an approximate 24-hour latency in the reflection of costs.)

2. AWS Lambda: The core of our automation, this function is triggered by EventBridge, queries AWS Cost Explorer for the current month's costs, and sends the cost details through email and Microsoft Teams.

3. AWS Cost Explorer: Provides detailed cost and usage data, which the Lambda function queries to retrieve the necessary information.

4. AWS SES (Simple Email Service): Used by the Lambda function to send the cost report via email.

5. Microsoft Teams: Receives the cost report through a channel notification.

Workflow

1. Scheduling the Event: Every day at 7:00 AM, AWS EventBridge triggers the Lambda function.
2. Lambda Function Execution:
   • Querying Costs: The Lambda function queries AWS Cost Explorer to get the current month's cost data.
   • Sending Notifications: The Lambda function then sends this cost data via two methods:
     • Email: Using AWS SES, the cost report is emailed to the designated recipients.
     • Microsoft Teams: The cost report is also sent to a specified Teams channel for quick access and visibility by the entire team.

Benefits

• Time-Saving: Automating the cost reporting process saves time and effort, eliminating the need for manual checks.
• Accuracy: Automated queries ensure that the cost data is consistently retrieved and reported without human error.
• Timeliness: Regular updates ensure that the team is always aware of the latest cost figures, aiding in better budget management and proactive cost control.

Conclusion

Implementing this automated cost reporting system has significantly improved our efficiency and cost management practices. By leveraging AWS services such as EventBridge, Lambda, Cost Explorer, SES, and integrating with Microsoft Teams, we now receive timely and accurate cost reports along with detailed information on which services were used and what charges were applied, all without manual intervention. This solution ensures that we stay on top of our AWS spending, making budget oversight easier and more effective.

Teams Channel Notification

Below is an example of a Teams channel notification generated by our automated cost reporting system. This notification is created using AWS Lambda and sent to the Teams channel using an adaptive card format

Coming Soon

Stay tuned for my next post, where I will share the Lambda code and provide a detailed guide on deploying this solution using AWS CDK.