DEV Community

Cover image for AMAZON S3 BRAIN DUMP
israel mvono
israel mvono

Posted on



  • Amazon S3 is one of the main building blocks of AWS.
  • It’s advertised as “infinitely scaling” storage.
  • Many websites use Amazon S3 as a backbone.

Amazon S3 Overview - Buckets

  • Amazon S3 allows people to store objects (files) in “buckets”(directories)
  • Buckets must have a globally unique name
  • Buckets are defined at the region level
  • Naming convention . No uppercase . No underscore . 3-63 characters long

Amazon S3 Overview - Objects

  • Objects (files) have a key.
  • The key is the FULL path:
    . S3://my-bucket/my_file.txt
    . S3://my-bucket/my_folder1/another_folder/my_file.txt

  • The key is composed of prefix + object name
    . S3://my-bucket/my_folder1/another_folder/my_file.txt

  • Object values are the content of the body:
    . Max object size is 5TB (5000 GB)
    . If uploading more than 5GB, must use “multi-part upload”

  • Metadata (list of text key / value pairs - system or user metadata)

  • Tags (unicode key / value pair - up to 10) - useful for security / lifecycle.

  • Version ID (if versioning is enabled)

Amazon S3 - Versioning

  • You can version your files in Amazon S3..
  • It is enabled at the bucket level.
  • Same key overwrite will increment the “version” : 1, 2, 3...
  • It is best practice to version your buckets.
    . Protect against unintended deletes(ability to restore a version)
    . Easy roll back to previous version

  • Any file that is not versioned prior to enable versioning will have version “null”

  • Suspending versioning does not delete the previous versions.

Amazon S3 Encryption for Objects

  • There are 4 methods of encrypting objects in S3 . SSE-S3: encrypts S3 objects using keys handled and managed by AWS . SSE-KMS: leverage AWS Key Management Service to manage encryption keys. . SSE-C: when you want to manage your own encrytion keys. . Client Side Encryption


  • SSE-S3: encryption using keys handled and managed by Amazon S3.
  • Object is encrypted Server Side.
  • AES-256 encryption type.
  • Must set header: “x-amz-server-side-encryption”:”AES256”


  • SSE-KMS: encryption using keys handled and managed by KMS.
  • KMS Advantages: user control + audit trail.
  • Object is encrypted Server Side.
  • Must set header: “x-amz-server-side-encryption”:”aws:kms”


  • SSE-C: Server side encryption using data keys fully managed by the customer outside of AWS.
  • Amazon S3 does not store the encryption key you provide.
  • HTTPS must be used.
  • Encryption key must be provided in the HTTP headers, for every HTTP request made.

Client Side Encryption

  • Client library such as the Amazon S3 Encrytion Client.
  • Clients must encrypt data themselves before sending to S3.
  • Clients must decrypt data themselves when retrieving from S3.
  • Customer fully manages the keys and encryption cycle.

Encryption in transit (SSL/TLS)

  • Amazon S3 exposes:
    . HTTP endpoint: non encrypted
    . HTTPS endpoint: encryprion in flight

  • You are free to use the endpoint you want but HTTPS is recommended.

  • Most clients would use the HTTPS endpoint by default.

  • HTTPS is mandatory for SSE-C.

  • Encryption in flight is also called SSL/TLS.

S3 Security
. User based

  • IAM policies - which API calls should be allowed for a specific user from IAM console.

. Resource based
. Bucket policies - bucket wide rules from the S3 console - allows cross account.
. Object Access Control List (ACL) - finer grain
. Bucket Access Control List (ACL) - less common

S3 Bucket Policies
. JSON based policies
. Resources: buckets and objects
. Actions: set of API to Allow or Deny
. Effect: Allow/ Deny
. Principal: the account or user to apply the policy to.

“version” : “2022-10-18”
“Statement” : [
“Sid” : “PublicRead”,
“Effect” : “Allow”,
“Principal” : “*”,
“Action” : [
“S3: GetObject”
“Resource” : [
“arn:aws:S3:::examplebucket / * “

. Use S3 bucket for policy to:
. Grant public access to the bucket.
. Force objects to be encrypted at upload.
. Grant access to another account (cross Account)

Bucket Settings for Block Public Access

  • Block public access to buckets and objects granted through: . new access control lists (ACLs) . any access control lists (ACLs) . new public bucket or access point policies
  • Block public and cross-account access to buckets and objects through any public bucket or access point policies.
  • These settings were created to prevent company data leaks.
  • If you know your bucket should never be public, leave these on.

S3 Security - Other
. Networking
. Supports VPC Endpoints (for instances in VPC without www internet)
. Logging and Audit:
. S3 Access Logs can be stored in other S3 bucket.
. API calls can be logged in AWS cloudTrail.
. User Security:
. MFA Delete: MFA( multi factor authentication) can be required in versioned buckets to delete objects.
. Pre-signed URLs: URLs that are valid only fro a limited time (ex: premium video service for logged in users)

CORS - Explained


  • If a client does a cross-origin request on your S3 bucket, you need to enable the correct CORS headers.
  • You can allow for a specific origin or for * (all origins)

Top comments (0)