Snowflake Feature Alert: Budgets

Learn to Manage Your Control Your Costs of Snowflake through the New Feature #budgets

Photo by Alexander Grey on Unsplash

Snowflake continues to improve how Cost Control happens in Snowflake. This time, they released a feature called Budgets.

What Are Budgets?

Budgets can monitor credit usage by setting monthly spending limits on compute costs for specific Snowflake objects or at an account level. 💰💳💻📈 When the budget exceeds the limit, then an email alert is sent to an email you designated. Budgets require the 2023_06 Bundle.

Monitoring budgets can include Tables, Materialized views, Schemas, Databases, Warehouses, Pipes, and Tasks. It is great coverage of objects to start, with most usage coming from these objects, outside of this, the only glaring miss is maybe Functions (UDFs), Apps, and Containers.

How To Use It

This is how you set up at an account level. You can see I set a budget of $400 dollars for this main account.

--Verify Bundle
select SYSTEM$BEHAVIOR_CHANGE_BUNDLE_STATUS('2023_06');

-- Activate the Account Budget, Set Limit 
CALL snowflake.local.account_root_budget!ACTIVATE();
CALL snowflake.local.account_root_budget!SET_SPENDING_LIMIT(400); -- Set the appropriate limit in credits

-- Create Notification Integration for budget alert
CREATE NOTIFICATION INTEGRATION budgets_notification_integration
  TYPE=EMAIL
  ENABLED=TRUE
  ALLOWED_RECIPIENTS=('yoda@example.com');

  -- Grant the USAGE privilege on the notification integration
GRANT USAGE ON INTEGRATION budgets_notification_integration
    TO APPLICATION snowflake;

-- Set the email notification for the account budget
CALL snowflake.local.account_root_budget!SET_EMAIL_NOTIFICATIONS(
   'budgets_notification_integration',
   'yoda@example.com');

This would allow a more granular budget targeting warehouses and pipes.

In my case, I want to monitor our Kafka pipeline and our LLM warehouse.

-- Activate the Account Budget, Set Limit at 400 credits
CALL snowflake.local.account_root_budget!ACTIVATE();
CALL snowflake.local.account_root_budget!SET_SPENDING_LIMIT(400); -- Set the appropriate limit in credits

-- Create Notification Integration for budget alert
CREATE NOTIFICATION INTEGRATION budgets_notification_integration
  TYPE=EMAIL
  ENABLED=TRUE
  ALLOWED_RECIPIENTS=('augusto@infostrux.com');

  -- Grant the USAGE privilege on the notification integration
GRANT USAGE ON INTEGRATION budgets_notification_integration
    TO APPLICATION snowflake;

-- Set the email notification for the account budget
CALL snowflake.local.account_root_budget!SET_EMAIL_NOTIFICATIONS(
   'budgets_notification_integration',
   'augusto@infostrux.com');

-- Lets create custom budgets for Warehouses and Pipes
-- Create a custom budget
USE SCHEMA cost_control.cost_control;
CREATE SNOWFLAKE.CORE.BUDGET custom_budget();

-- Set spending limit at 100 Credits
CALL custom_budget!SET_SPENDING_LIMIT(100);

-- Set email
CALL custom_budget!SET_EMAIL_NOTIFICATIONS(
   'budgets_notification_integration',
   'augusto@infostrux.com'
);

-- Add objects to the custom budget
CALL cost_control.cost_control.custom_budget!ADD_RESOURCE(
   SYSTEM$REFERENCE('DATABASE', 'COST_CONTROL', 'SESSION', 'applybudget'));

-- Add LLM warehouse
CALL cost_control.cost_control.custom_budget!ADD_RESOURCE(
   SYSTEM$REFERENCE('WAREHOUSE', 'LLM_WH_XXL', 'SESSION', 'applybudget'));

-- Add snowpipe pipe
CALL cost_control.cost_control.custom_budget!ADD_RESOURCE(
   SYSTEM$REFERENCE('PIPE', 'infostrux_snowpipe', 'SESSION', 'applybudget'));

You can see how easy it is to add budgets to other objects.

What does the UI look like?

UI looks nice, and you can access it by going to Admin > Usage > 2nd tab after Consumption.

Right below this graph, you can see custom budgets and percentage of use.

Where is how to edit the account budget

What Is Missing?

At Infostrux, we have many Snowflake accounts, all billing through Snowflake Organization to the same account. It would be nice to have this function across accounts and set up Budgets across all accounts in the main report.

The UI should probably break down all three chargeable components (storage, egress, and compute), and having the ability to see dollars and credits as a choice would be nice.

Check out other posted articles about Cost Control in the last few months:

• https://blog.infostrux.com/costs-control-series-part-1-how-to-save-money-with-snowflake-459438603882
• https://blog.infostrux.com/cost-control-series-part-2-snowflake-best-practices-a16d6403f550

Conclusion

Snowflake should have done this a long time ago, but with economic pressures, they are finally paying attention to cost management, so this is quite nice. Just FYI, most things I need via SQL are also available via the Snowflake UI.

I think this is a great start, although I feel it should have included Snowflake Organizations. It is quite a challenge to manage multiple accounts' cost control at the moment, although it is probably not a problem most companies deal with.

Thank you, Snowflake Product team, for releasing this feature.

I’m Augusto Rosa, VP of Engineering for Infostrux Solutions. Thanks for reading my blog post. You can follow me on LinkedIn. Subscribe to Infostrux Medium Blogs https://medium.com/infostrux-solutions for the most interesting Data Engineering and Snowflake news.

Sources:

• https://docs.snowflake.com/en/user-guide/budgets