DEV Community

Rajit Paul
Rajit Paul

Posted on

Cross Account S3 Bucket Migration

Cross Account Migration of S3 Bucket

Hi Folks,
Today we are going to look into another usecase of AWS - Cross Account Migration of an S3 Bucket.

Before we get started,

What is S3?

S3 - Simple Storage Service, is an object storage in AWS used to store images, videos and other object related storage. S3 Standard Storage class has a durability of 11 9's and availability of 99.99/year.

What are S3 Buckets?

S3 Buckets - Basic unit of storage in S3, each bucket in S3 can store up to 5TB of data.

What are S3 Bucket Policy?

S3 Bucket Policy - Resource based AWS IAM Policy. Used to grant other AWS Accounts or IAM users access to the bucket.

What is IAM?

IAM - Identity and Access Management - Helps to manage access to AWS services and resources securely. Via IAM we can create users, policies and roles respectively.

Now let us proceed with the use-case.

~Pre-Requisites-

2 AWS Accounts
2 S3 Buckets (One Source Bucket & One Destination Bucket)
2 users with S3 Bucket Creation, Bucket Policy Creation and IAM Privileges.

First Step - Copy the Account Number of the destination account, we shall require that in the next step.

2. Source Side changes

Source S3 Bucket - Bucket1

Bucket 1 Policy
-
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "CrossAccountS3Access",
"Effect": "Allow",
"Principal": {"AWS": "111111111111"},
"Action": ["s3:ListBucket","s3:GetObject"],
"Resource": [
"arn:aws:s3:::SOURCE-BUCKET-NAME/*",
"arn:aws:s3:::SOURCE-BUCKET-NAME"
]
}
]
}
Note: Please replace the Account No(111111111111) with your Destination Account Number, and the SOURCE-BUCKET-NAME with the Source S3 Bucket Name.

3.Create an IAM User and attach a policy to the IAM user

  • Create an user on the IAM Console, we will consider the IAM username to be Destination User.
  • Create a policy for the user named Destination User Policy

{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:ListBucket",
"s3:GetObject"
],
"Resource": [
"arn:aws:s3:::SOURCE-BUCKET-NAME/ *",
"arn:aws:s3:::SOURCE-BUCKET-NAME"

]
},
{
"Effect": "Allow",
"Action": [
"s3:ListBucket",
"s3:PutObject",
"s3:PutObjectAcl"
],
"Resource": [
"arn:aws:s3:::DESTINATION-BUCKET-NAME/ *",
"arn:aws:s3:::DESTINATION-BUCKET-NAME"
]
}
]
}

Note: Replace SOURCE-BUCKET-NAME & DESTINATION-BUCKET-NAME with your Source and Destination Bucket name respectively.

4. Sync S3 Bucket from Source to Destination

aws s3 sync s3://SOURCE-BUCKET-NAME s3://DESTINATION-BUCKET-NAME --source-region SOURCE_REGION_NAME --region DESTINATION_REGION_NAME

Note: Please replace the SOURCE-BUCKET-NAME, DESTINATION-BUCKET-NAME, SOURCE_REGION_NAME, DESTINATION_REGION_NAME with your Source Bucket Name, Destination Bucket Name, Source Region Name, Destination Region Name respectively.

This brings us to the end of this blog, have a nice day, cheers.

Discussion (0)