How to migrate data between two S3 buckets in different accounts with SSO user access?

In this blog, I am going to explain how to transfer objects from one S3 bucket present in one AWS account to another S3 bucket in another account, with SSO user privilege. Both the buckets are created in different regions.

Prerequisites

Two AWS accounts (One with the source S3 bucket and another with the destination S3 bucket)
SSO user access.

What is SSO?

Single sign-on (SSO) is defined to be an authentication solution which allows users to log-in to multiple applications and websites with a one-time user authentication.

Once the user has been given the access, the log-in page will look something like this -

Step — 1

Sign-in to the AWS account with the source bucket.

Go to the bucket that has to be copied(source bucket), and then we have to edit the bucket policy.

The policy to be used is as follows -

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "DelegateS3Access",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::destinationaccountnumber:root"
            },
            "Action": [
                "s3:ListBucket",
                "s3:GetObject"
            ],
            "Resource": [
                "arn:aws:s3:::sourcebucketname/*",
                "arn:aws:s3:::sourcebucketname"
            ]
        }
    ]
}

Step — 2

Log-in to the AWS account where the data will be transferred to, and create a policy -

The policy to be used is as follows -

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket",
                "s3:GetObject"
            ],
            "Resource": [
                "arn:aws:s3:::sourcebucketname",
                "arn:aws:s3:::sourcebucketname/*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket",
                "s3:PutObject",
                "s3:PutObjectAcl"
            ],
            "Resource": [
                "arn:aws:s3:::destinationbucketname",
                "arn:aws:s3:::destinationbucketname/*"
            ]
        }
    ]
}

Step — 3

Add this policy to a role, and add the role to a user.

Step — 4

Next, go to your terminal, and configure your terminal with the required credentials of the newly created user from your SSO log-in page.

Check if you are able to access the AWS account with the destination bucket with the following command -

aws sts get-caller-identity

Once you are able to access your AWS account, the next step is to check if you are able to access the source bucket from this AWS account.

aws s3 ls s3://sourcebucketname

Step — 5

Next, we have to copy the data from the source bucket into the destination bucket. We have to use the following command —

aws s3 sync s3://sourcebucketname s3://destinationbucketname — source-region sourceregion — region destinationregion

Conclusion —

Thus, in a few steps, we will be able to copy data from one S3 bucket to another S3 bucket belonging to a completely different account and a different region.

One concern for this method of data transfer is the size of the bucket. If the bucket size is large, then this way of data transfer will require a much longer time.

Hope you find this blog helpful.

Happy learning!!