We can replicate encrypted S3 objects from one region to another. Copying objects has a couple of prerequisites. One such step is giving relevant permissions to S3.
1. Cross-region replication
Some AWS users want to replicate their S3 objects to another region for audit or backup reasons.
Cross-region replication is a well-known strategy which copies objects within minutes to a different bucket in another AWS region.
This post won't go into details on how to configure cross-region replication. I'll have a link at the bottom of the page that helps set up the operation.
Since the purpose of this article is to discuss permissions, I'm only mentioning the most important steps of setting up cross-region replication here.
1.1. Versioning
We must enable versioning if we want to copy the objects to another bucket. Replication won't work without versioning.
It's also worth mentioning that versioning and replication are not retrospective. The automation won't copy objects already existing in the source bucket. There is a way to copy these objects through batch operations but it's different from the automatic copy mechanism.
1.2. Source and destination buckets
We should have two buckets: one is the source and another is the destination. In this example, we will have the source bucket in us-east-1 and the destination bucket in us-west-2.
1.3. Encrypted objects
We'll apply server-side encryption with a KMS-managed key. We'll have to create a new key in us-east-1 that S3 uses to encrypt the objects.
For the purposes of this article we'll use a multi-region key. We should then create a replica of the KMS-key in us-west-2.
More on the multi-region key later below.
1.4. Role
S3 will need an IAM role to perform the object replication.
The replication will fail if we don't assign the role proper permissions. S3 needs to access, replicate, encrypt and decrypt the objects.
2. Permissions
It's best to create a new role that we dedicate to cross-region replication. It is because we can limit the permissions to the minimum necessary level as per the least privilege principle.
2.1. Trust policy
The role should have a trust policy that allows S3 to assume it. It will look like this:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": "s3.amazonaws.com"
},
"Action": "sts:AssumeRole"
}
]
}
S3 will use the role to perform the replication, so we'll have to add the relevant permissions to it.
2.2. Permissions on the source bucket
S3 will have to access the objects in the source bucket. We will have to allow it to do so:
{
"Effect": "Allow",
"Action": [
"s3:GetObjectVersionTagging",
"s3:GetObjectVersionAcl",
"s3:ListBucket",
"s3:GetObjectVersionForReplication",
"s3:GetReplicationConfiguration"
],
"Resource": [
"arn:aws:s3:::SOURCE_BUCKET_NAME/*",
"arn:aws:s3:::SOURCE_BUCKET_NAME",
]
},
The s3:GetObjectVersionForReplication and s3:GetReplicationConfiguration permissions are probably the most interesting ones here. Both are related to replication. They also show that we must enable versioning for the operation.
2.3. Permissions on the destination bucket
The role should have permission to replicate the objects to the destination bucket:
{
"Effect": "Allow",
"Action": [
"s3:ReplicateObject",
"s3:ReplicateTags",
"s3:ReplicateDelete"
],
"Resource": "arn:aws:s3:::DESTINATION_BUCKET_NAME/*"
}
AWS recommends these permissions for copying the objects to the destination bucket.
2.4. Encryption and decryption
We have encrypted the objects with a KMS key.
S3 can't copy objects in an encrypted format. It will first need to decrypt, then re-encrypt them before saving them to the destination bucket.
It means that we must give S3 kms:Decrypt permission to the key in the source region, which is us-east-1:
{
"Effect": "Allow",
"Action": [
"kms:Decrypt",
],
"Resource": [
"arn:aws:kms:us-east-1:ACCOUNT_NUMBER:key/mrk-aaaaaaaaaaaaaaaa1111111111111111",
]
}
The destination bucket is in us-west-2, so the policy must contain the kms:Encrypt permission to the key in that region:
{
"Effect": "Allow",
"Action": [
"kms:Encrypt",
],
"Resource": [
"arn:aws:kms:us-west-2:ACCOUNT_NUMBER:key/mrk-aaaaaaaaaaaaaaaa1111111111111111"
]
},
That's it! If we now upload a file to the source bucket in us-east-1, it should appear in the destination bucket in us-west-2 a few seconds later.
The objects in the source bucket will remain encrypted, and S3 have encrypted the copies in the destination bucket too.
3. Multi-region keys for cross-region replication
The IDs of the two KMS keys (mrk-xxx...) are the same. They indicate that we used multi-region keys to encrypt objects in both buckets.
A multi-region key consists of the original key (in us-east-1 in this case) and its replicas in other regions.
One of the advantages of a multi-region key is that we don't need a different key to re-encrypt data in the other region. This feature will reduce latency and make the encryption less complex.
But this feature doesn't work for S3 cross-region replication as of writing this. S3 will treat the multi-region keys as two different, single-region keys. As opposed to DynamoDB global tables where the multi-region key encrypted items get copied, S3 will first decrypt the object, then copy it and then finally re-encrypt it. We must give the role that S3 assumes permission to both the original key (kms:Decrypt) and its replica (kms:Encrypt).
4. Summary
Cross-region replication of S3 objects is a widely-known strategy. To copy them from one region to another, S3 must assume a role that gives it the relevant permissions.
When we want to replicate server-side KMS-encrypted objects, we will not only give replication permissions to the role but kms:Decrypt and kms:Encrypt permissions too. The rule is the same for multi-region KMS keys.
5. Further reading
Setting up replication - How to set up cross-region replication in S3
Replicating encrypted objects - Step-by-step guide for replicating encrypted objects in the Console and CLI
Multi-Region keys in AWS KMS - Details about multi-region keys
Top comments (0)