When you buy a t-shirt at the store, you (generally) pay the same price for the shirt if you are an XS or a 2XL. In the cloud, the size of the Virtual machine you select has VERY large implications in the cost. At AWS, launching a t2.nano instance costs $.0058 an hour – or 14 cents a day, while a t3.2xlarge instance is $.4628 – or $11.11 a day.
When sizing a new system, it is common to go a “bit larger” in size to ensure that the service performs well. This has worked well for AWS, as over 50% of their revenue comes from EC2 instances. In a recent post, we built an automated RunBook to examine our daily spend for each AWS product, and EC2 (the green line) is far and away our biggest cost center:
When looking at your highest cost center – you may wish to drill deeper into what these expenses are. In this post, we’ll break down our daily EC2 spend by the types of instances that are running.
Creating the RunBook
In our last post, we built a RunBook that used the AWS Cost and Usage report to break down unSkript’s costs product. By running the report daily, we could chart our daily cost spend per product, and if the change day over day exceeded a threshold, we could send an alert.
To study our daily EC2 usage and spend, we will use the same Cost and Usage report, and the RunBook is essentially the same – just with a different SQL query into the table. Rather than recreate the RunBook, we will simply Duplicate the RunBook in the unSkript UI, and save it with a new name:
To run a AWS Redshift query, we need a SecretArn, the SQL Query, AWS Region and Redshift Cluster and database details.
The RunBook generates these all for us, but we need to update the SqlQuery variable to query EC2 data:
sqlQuery = f"SELECT date_part(day, cast(lineitem_usagestartdate as date)) as day, product_instancetype,SUM(lineitem_usageamount)::numeric(37, 4) AS usage_hours, SUM((lineitem_unblendedcost)::numeric(37,4)) AS usage_cost FROM {tableName} WHERE length(lineitem_usagestartdate)>8 AND product_productfamily = 'Compute Instance' AND pricing_unit IN ('Hours', 'Hrs') GROUP BY day, product_instancetype ORDER BY 1 DESC, 3 DESC, 2
The next few steps of this RunBook is unchanged from the Product Cost RunBook:
We take the Ann and the query – and run the query – then pull the results into the RunBook.
We must now change the chart – since the inputs are different. Our Columns are now “product_instancetype” and the y values are “usage_cost.”
Charting this data helps us see which instance types are costing the most money:
Our daily costs are very much proportional to size, with the 2xLarge and Large instances accounting for ~$20/day. It is interesting to note that earlier this month, our t2.large went down at the same time our t2.micro grew – this could possibly be interpreted as resizing of an EC2 instance that was too large.
Finally, we can build alerts to tell us if any of our costs jump by over $1 a day, or 10%. If the costs make a jump we can send the chart EC2 usage to Slack using the Send Image To Slack Action (coming this week to our Open Source). We also send this image every Monday for a historical record:
Conclusion
When it comes to FinOps – the team in charge of understanding and accounting for your cloud bill, one of the most critical tools to have is observability into your daily spend. If there is a large change – that might be ok – but it is good to know in advance, and double check that the increase is valid – thereby potentially avoiding a large bill at the end of the month.
In our previous post, we looked at all AWS products for daily cost spend, and in this post, we dug deeper to better understand our EC2 spend – the largest percentage of our AWS bill. Are you interested in trying out our alerting in unSkript? Try out our Open Source Docker build (and give us a star!), or sign up for a free cloud trial. Is there a segment of YOUR AWS bill that you’d like to investigate with us? Reach out in our Slack channel, and we’d be happy to help you create a RunBook for your use case!
Top comments (0)