Amazon S3 Cheat Sheet

Features

• Amazon S3 allows people to store objects (files) in “buckets” (directories)
• Buckets must have a globally unique name
  • Naming convention:
    • No uppercase
    • No underscore
    • 3-63 characters long
    • Not an IP
    • Must start with lowercase letter or number
• Objects
  • Objects (files) have a Key. The key is the FULL path:
    • /my_file.txt
    • /my_folder/another_folder/my_file.txt
  • There’s no concept of “directories” within buckets (although the UI will trick you to think otherwise)
  • Just keys with very long names that contain slashes (“/“)
  • Object Values are the content of the body:
    • Max Size is 5TB
    • If uploading more than 5GB, must use “multi-part upload”
  • Metadata (list of text key / value pairs - system or user metadata)
  • Tags (Unicode key / value pair - up to 10) - useful for security / lifecycle
  • Version ID (if versioning

Versioning

• It is enabled at the bucket level
• Same key overwrite will increment the “version”: 1, 2, 3
• It is best practice to version your buckets
  • Protect against unintended deletes (ability to restore a version)
  • Easy roll back to previous versions
• Any file that is not version prior to enabling versioning will have the version “null”

Encryption for Objects

• There are 4 methods of encrypt objects in S3
  • SSE-S3: encrypts S3 objects
    • Encryption using keys handled & managed by AWS S3
    • Object is encrypted server side
    • AES-256 encryption type
    • Must set header: “x-amz-server-side-encryption”:”AES256”
  • SSE-KMS: encryption using keys handled & managed by KMS
    • KMS Advantages: user control + audit trail
    • Object is encrypted server side
    • Maintain control of the rotation policy for the encryption keys
    • Must set header: “x-amz-server-side-encryption”:”aws:kms”
  • SSE-C: server-side encryption using data keys fully managed by the customer outside of AWS
    • Amazon S3 does not store the encryption key you provide
    • HTTPS must be used
    • Encryption key must be provided in HTTP headers, for every HTTP request made
  • Client Side Encryption
    • Client library such as the amazon S3 Encryption Client
    • Clients must encrypt data themselves before sending to S3
    • Clients must decrypt data themselves when retrieving from S3
    • Customer fully manages the keys and encryption cycle

Encryption in transit (SSL)

• exposes:
  • HTTP endpoint: non encrypted
  • HTTPS endpoint: encryption in flight
• You’re free to use the endpoint your ant, but HTTPS is recommended
• HTTPS is mandatory for SSE-C
• Encryption in flight is also called SSL / TLS

Security

By default, all S3 objects are private

A user who does not have AWS credentials or permission to access an S3 object can be granted temporary access by using a presigned URL. A presigned URL is generated by an AWS user who has access to the object. The generated URL is then given to the unauthorized user

• User based
  • IAM policies - which API calls should be allowed for a specific user from IAM console
• Resource based
  • Bucket policies - bucket wide rules from the S3 console - allows cross account
  • Object Access Control List (ACL) - finer grain
  • Bucket Access Control List (ACL) - less common
• Networking
  • Support VPC endpoints (for instances in VPC without www internet)
• Logging and Audit:
  • S3 access logs can be stored in other S3 buckets
  • API calls can be logged in AWS CloudTrail
• User Security:
• MFA (multi factor authentication) can be required in versioned buckets to delete objects
• Signed URLs: URLS that are valid only for a limited time (ex: premium video services for logged in users)

• Practice notes
• Questions