“ I have checked the documents of AWS to create a glue database, table and partition parameters easily in a multi account scenario without any serverless function . So I checked for various solutions and found out that it can be possible using cloudformation stack. Pricing of S3 bucket as per storage size and glue costs cheaper.”

AWS Glue is a serverless data integration service that makes it easy to discover, prepare, and combine data for analytics, machine learning, and application development. AWS Glue provides all of the capabilities needed for data integration so that you can start analyzing your data and putting it to use in minutes instead of months. AWS Glue provides both visual and code-based interfaces to make data integration easier. Users can easily find and access data using the AWS Glue Data Catalog.

In this post, you will get to know how to explore customized glue parameters! Database, table and partition through cloudformation. Here I have created a s3 bucket, glue through a cloudformation stack with customized parameters.

Prerequisites

You’ll need an Amazon Simple Storage Service for this post. Getting started with Amazon Simple Storage Service provides instructions on how to create a bucket in simple storage service.

Architecture Overview

The architecture diagram shows the overall deployment architecture with data flow, amazon s3, aws glue and cloudformation.

Solution overview

The blog post consists of the following phases:

1. Create of Glue Database, Table and Partition through Cloudformation
2. Output of Stack “AWS Glue”

I have a s3 bucket as below →

Phase 1: Create of Glue Database, Table and Partition through Cloudformation

1. Open the cloudformation console and create a stack for the glue database and table. In the template file, various parameters are defined such as: database parameters, table(schema) parameters and partition parameters.

Phase 2: Output of Stack “AWS Glue”

1. Open the glue console and verify the database, table and partition created.

Clean-up

Delete S3 Bucket and Cloudformation Stack.

Pricing

I review the pricing and estimated cost of this example.

Cost of Data Transfer = $0.0

Cost of Glue = $0.0

Cost of S3 = $0.1

Total Cost = $0.1

Summary

In this post, I showed “how to explore customized glue parameters! Database, table and partition through cloudformation”.

For more details on AWS Glue, Checkout Get started AWS Glue, open the AWS Glue console. To learn more, read the AWS Glue documentation.

Thanks for reading!

Connect with me: Linkedin