As mentioned in last part
- We already have a S3 bucket which is Amplify set up for us
- Now when a file uploaded to that bucket, we need to notify a lambda function, which we will use other Aws service inside it
What is Amazon EventBridge?
Amazon EventBridge is a serverless event bus service that makes it easy to connect your applications with data from a variety of sources. EventBridge was formerly called Amazon CloudWatch Events. (From aws docs)
Based on my understanding:
- Anything happen in your resource, will emit an
events
-
events
is actually a log files in json format produce by CloudTrail. - EventBridge is the watcher for this
events
- You tell EventBrigde about what
events
you want to watch(In our case, when a file uploaded to our S3 bucket) - You tell EventBridge when that
events
happen what you want to do(In our case, invoke a Lambda Function)
What we need TODO:
- Define S3 bucket to store the
event
(log files produce by CloudTrail) - Define the bucket policy for that S3 bucket (Bucket policy)
- Tell EventBridge what pattern of
event
you want to watch (EventRule) - Tell EventBridge if that pattern of
event
happened, what you want it to do. (EventTarget)
Ok, how we do that?
In this blog post, we will provisioned all the resource defined above using AWS CDK.
Why we use CDK?
Think about this, AWS have 160+ services. Every time when we need something, we have to define it 1 by 1 in the AWS management console. Very time consuming, need to find service by service in the console.
No, we don't do that. We write some code using AWS CDK. Then we run a short command cdk deploy
, then everything get set up in the cloud.
Nice.
Let's get started
First set up the cdk project using Python:
Make a directory
$ mkdir mycdkapp && cd mycdkapp
Set up a CDK app
$ cdk init app --language python
Activate the virtual environment
$ .env\Scripts\activate.bat
Install all the dependencies needed
$ pip install -r requirements.txt
Cool, so we done setting up a CDK app for our project.
Now you will see mycdkapp_stack.py
in your directory. This is where we define all the resource need here.
Refer to the existing S3 bucket and create a new Logging bucket for CloudTrail
Install the library needed:
$ pip install aws_cdk.aws_s3
This will install the L1 construct
which represent a single type of aws service. All this create by aws, so every time you need a particular service, you can find the library here
In this case, we install the aws_s3
library.
Now paste the code below into mycdkapp_stack.py
from aws_cdk import (
aws_s3 as s3,
core
)
class MycdkappStack(core.Stack):
def __init__(self, scope: core.Construct, id: str, **kwargs) -> None:
super().__init__(scope, id, **kwargs)
# refer the existing s3 bucket, substitute <your-existing-s3-bucket-arn> with your bucket arn
existing_bucket=s3.Bucket.from_bucket_arn(self, "BucketByArn", "arn:aws:s3:::<your-existing-s3-bucket-arn>")
# this create a new s3 bucket with name s3LoggingBucket
logging_bucket = s3.Bucket(self,"s3LoggingBucket", bucket_name="s3loggingbucket")
With just 2 lines of code, we already get the existing_bucket
and created a new bucket named s3LoggingBucket
.
Now run this
cdk synth > template.yaml
You will see a template.yaml in your directory.
Next we need to set up Bucket policy for this Logging bucket.
- Make a new directory name
contruct
insidemycdkapp
- Create a new file named
logging_policy.py
-
pip install aws_cdk.aws_iam
- Paste this below code into
logging_policy.py
from aws_cdk import (
aws_s3 as s3,
aws_iam as iam,
core,
)
class LoggingBucketPolicy(core.Construct):
def __init__(self,scope: core.Construct, id: str, _bucket: s3.IBucket, **kwargs):
super().__init__(scope,id, **kwargs)
cloudtrail_service_principal = iam.ServicePrincipal('cloudtrail.amazonaws.com')
get_bucket_policy = iam.PolicyStatement(
actions=["s3:GetBucketAcl"],
resources=[_bucket.bucket_arn],
principals=[cloudtrail_service_principal]
)
put_object_policy = iam.PolicyStatement(
actions=['s3:PutObject'],
resources=[f"arn:aws:s3:::{_bucket.bucket_name}/AWSLogs/{core.Environment.account}/*"],
principals=[cloudtrail_service_principal]
)
_bucket.add_to_resource_policy(get_bucket_policy)
_bucket.add_to_resource_policy(put_object_policy)
What is this code above do?
- We define a
Construct
by inheritcore.Construct
- This let us group together all the component which doing the same thing(In this case, this construct will define policy to a bucket)
- We will insert
s3.IBucket
into thisconstruct
with variable_bucket
- Define 2
iam
policyget_object_policy
andput_object_policy
- Allow both policy can use by
cloudtrail.amazonaws.com
- Then the policy to the
_bucket
Now in our mycdkapp_stack.py
add this line:
LoggingBucketPolicy(self, 's3Policy', logging_bucket)
Then run
cdk synth > template.yaml
Now check template.yaml
, you will see something like this:
s3LoggingBucketPolicyEF0E9CBA:
Type: AWS::S3::BucketPolicy
Properties:
Bucket:
Ref: s3LoggingBucket06C9F6F3
PolicyDocument:
Statement:
- Action: s3:GetBucketAcl
Effect: Allow
Principal:
Service: cloudtrail.amazonaws.com
Resource:
Done.
*Now you successfully: *
- Understand what is Amazon Eventbridge and why we use it.
- Understand what is Aws CDK and why we use it.
- Refer the existing bucket into our newly created cdk app.
- Create a new Logging bucket for CloudTrail using CDK
- Apply all bucket policy to the new bucket using CDK
*Next we will talking about, *
- How to define a Event rule/pattern for Amazon Eventbrige?
- When the certain pattern in Eventbridge occurred, what should do next?
All using CDK.
Stay tuned.
[Part 1]What is Amazon EventBridge and How to provision Logging S3 bucket, and bucket policy using AwsCDK
Coming soon
[Part 2] How to define EventRule, EventTarget for Amazon Eventbridge using CDK.
Before you go, if you like this series or find this useful consider to buy me a coffee 😊🤞 for 5 USD or more.
I will prepare a GitHub repo for this whole tutorial series and arrange into separate commit for each part.
This will only available for my supporter cause I spent a lot of time to prepare this. Anyway, I appreciate you here. Have a good day.
Follow me on Twitter: @upupkenchoong
My upcoming product(If you interested): @sarah_assistant
Top comments (0)