DEV Community

Adeline Makokha for AWS Community Builders

Posted on

AMAZON STORAGE GATEWAY

YOU HAVE DATA AND YOU WANT TO STORE IT IN THE CLOUD AND ACCESS IT ON-PREM

a

WE`LL BE TALKING ABOUT DATA THAT IS:

  • CONNECTED TO ON-PREM APPLICATIONS
  • PART OF CONTINUOUS HYBRID WORKFLOWS
  • BACKUP FOR CRITICAL SYSTEMS
  • IN BULK STORAGE

MOVING DATA TO THE CLOUD?

b

WHAT IS STORAGE GATEWAY?
A HYBRID STORAGE VIRTUAL APPLIANCE DESIGNED TO RUN IN A VIRTUAL ENVIRONMENT ON-PREM.
A HARDWARE APPLIANCE CAN ALSO BE USED
Connects an on-premises software appliance with cloud-based storage to provide seamless integration between your on-premises IT environment and the AWS storage infrastructure.

STORAGE GATEWAY ARCHITECTURE

c

THE THREE KEY HYBRID USE CASE
1.Move backups and archives to the cloud.
2.Reduce on-premises storage with cloud-backed file shares.
3.Provide on-premises applications low latency access to data stored in AWS

d

STORAGE GATEWAY MODES

STORAGE GATEWAY CAN BE IMPLEMENTED IN THREE WAYS:

  • FILE MODE(FILE STORAGE BACKE DBY S3) – NFS & SMB
  • VOLUME MODE(CACHE/STORED – CLOUD VOLUMES RUNNING ON-PREM) - iSCSI
  • TAPE GATEWAY MODE(VIRTUAL TAPE LIBRARY MODE) - iSCSI

FILE GATEWAY

  • File Gateway supports a file interface into Amazon Simple Storage Service (Amazon S3) and combines a service and a virtual software appliance.
  • By using this combination, you can store and retrieve objects in Amazon S3 using industry-standard file protocols such as Network File System (NFS) and Server Message Block (SMB).

  • The software appliance, or gateway, is deployed into your on-premises environment as a virtual machine (VM) running on VMware ESXi, Microsoft Hyper-V, or Linux Kernel-based Virtual Machine (KVM) hypervisor.

  • The gateway provides access to objects in S3 as files.

FILE GATEWAY ARCHITECTURE
m

FILE GATEWAY
n

h

WITH FILE GATEWAY

  • You can store and retrieve files directly using the NFS version 3 or 4.1 protocol.
  • You can store and retrieve files directly using the SMB file system version, 2 and 3 protocol.
  • You can access your data directly in Amazon S3 from any AWS Cloud application or service.
    -You can manage your S3 data using lifecycle policies, cross-region replication, and versioning. You can think of a file gateway as a file system mount on S3.
    HOW TO USE THE FILE GATEWAY

  • TO USE A FILE GATEWAY, YOU START BY DOWNLOADING A VM IMAGE FOR THE FILE GATEWAY.

  • YOU THEN ACTIVATE THE FILE GATEWAY FROM THE AWS MANAGEMENT CONSOLE OR THROUGH THE STORAGE GATEWAY API.

  • AFTER THE FILE GATEWAY IS ACTIVATED, YOU CREATE AND CONFIGURE YOUR FILE SHARE AND ASSOCIATE THAT SHARE WITHYOUR AMAZON SIMPLE STORAGE SERVICE (AMAZON S3) BUCKET.

  • DOING THIS MAKES THE SHARE ACCESSIBLE BY CLIENTS USING EITHER THE NETWORK FILE SYSTEM (NFS) OR SERVER MESSAGE BLOCK (SMB) PROTOCOL.

  • FILES WRITTEN TO A FILE SHARE BECOME OBJECTS IN AMAZON S3.

  • OBJECTS ARE ENCRYPTED WITH AMAZON S3–SERVER-SIDE ENCRYPTION KEYS (SSE-S3). ALL DATA TRANSFER IS DONE THROUGH HTTPS.

VOLUME GATEWAY

  • A VOLUME GATEWAY PROVIDES CLOUD-BACKED STORAGE VOLUMES THAT YOU CAN MOUNT AS INTERNET SMALL COMPUTER SYSTEM INTERFACE (ISCSI) DEVICES FROM YOUR ON-PREMISES APPLICATION SERVERS.
  • THE VOLUME GATEWAY IS DEPLOYED INTO YOUR ON-PREMISES ENVIRONMENT AS A VM.

VOLUME GATEWAY IS IMPLEMENTED IN TWO MODES:

  1. STORED VOLUMES
  2. CACHED VOLUMES

STORED VOLUMES

  • BY USING STORED VOLUMES, YOU CAN STORE YOUR PRIMARY DATA LOCALLY, WHILE ASYNCHRONOUSLY BACKING UP THAT DATA TO AWS.

  • STORED VOLUMES PROVIDE YOUR ON-PREMISES APPLICATIONS WITH LOW-LATENCY ACCESS TO THEIR ENTIRE DATASETS. AT THE SAME TIME, THEY PROVIDE DURABLE, OFFSITE BACKUPS.

  • YOU CAN CREATE STORAGE VOLUMES AND MOUNT THEM AS ISCSI DEVICES FROM YOUR ON-PREMISES APPLICATION SERVERS.

  • DATA WRITTEN TO YOUR STORED VOLUMES IS STORED ON YOUR ON-PREMISES STORAGE HARDWARE.

  • THIS DATA IS ASYNCHRONOUSLY BACKED UP TO AMAZON S3 AS AMAZON ELASTIC BLOCK STORE (AMAZON EBS) SNAPSHOTS
    HOW TO IMPLEMENT

  • After you install the AWS Storage Gateway software appliance—the VM—on a host in your data center and activate it, you can create gateway storage volumes.

  • You then map them to on-premises direct attached storage (DAS) or storage area network (SAN) disks.

  • You can then mount these storage volumes to your on-premises application servers as iSCSI devices.

  • As your on-premises applications write data to and read data from a gateway's storage volume, this data is stored and retrieved from the volume's assigned disk. To prepare data for upload to Amazon S3, your gateway also stores incoming data in a staging area, referred to as an upload buffer. You can use on-premises DAS or SAN disks for working storage.

  • Your gateway uploads data from the upload buffer over an encrypted Secure Sockets Layer (SSL) connection to the AWS Storage Gateway service running in the AWS Cloud.

  • The service then stores the data encrypted in Amazon S3.

STORED VOLUMES ARCHITECTURE-MIGRATION AND DISASTER RECOVERY

m

STORED VOLUMES UNDER THE HOOD

n

STORED VOLUME GATEWAY
x

CACHED GATEWAY MODE

  • YOU STORE YOUR DATA IN AMAZON SIMPLE STORAGE SERVICE (AMAZON S3) AND RETAIN A COPY OF FREQUENTLY ACCESSED DATA SUBSETS LOCALLY.
  • CACHED VOLUMES OFFER A SUBSTANTIAL COST SAVINGS ON PRIMARY STORAGE AND MINIMIZE THE NEED TO SCALE YOUR STORAGE ON-PREMISES.
  • YOU ALSO RETAIN LOW-LATENCY ACCESS TO YOUR FREQUENTLY ACCESSED DATA.

CACHED VOLUMES-EXTEND LIMITED STORAGE CAPACITY INTO AWS

y

UNDER THE HOOD

g

CACHED MODE
s

TAPE GATEWAY MODE

  • A TAPE GATEWAY PROVIDES CLOUD-BACKED VIRTUAL TAPE STORAGE. THE TAPE GATEWAY IS DEPLOYED INTO YOUR ON-PREMISES ENVIRONMENT AS A VM
  • WITH A TAPE GATEWAY, YOU CAN COST-EFFECTIVELY AND DURABLY ARCHIVE BACKUP DATA IN GLACIER OR DEEP_ARCHIVE.
  • A TAPE GATEWAY PROVIDES A VIRTUAL TAPE INFRASTRUCTURE THAT SCALES SEAMLESSLY WITH YOUR BUSINESS NEEDS AND ELIMINATES THE OPERATIONAL BURDEN OF PROVISIONING, SCALING, AND MAINTAINING A PHYSICAL TAPE INFRASTRUCTURE.

TAPE GATEWAY ARCHITECTURE-MIGRATE A PHYSICAL TAPE ARCHITECTURE INTO AWS

b

TAPE GATEWAY
f

SUMMARY
g

v

Top comments (0)