Introduction
Managing traditional SFTP servers on-premises often comes with its share of challenges. Organizations struggle with maintaining the infrastructure, ensuring high availability, scaling storage, and securing user access. These systems require regular patching, upgrades, and constant monitoring
to prevent downtime or security breaches. For businesses handling increasing file transfer demands, these limitations can result in operational inefficiencies and spiraling costs.
Thankfully, AWS SFTP Transfer Family offers a modern solution to these issues. With its serverless and fully managed setup, you can eliminate the overhead of managing hardware while leveraging the scalability and cost-effectiveness of the AWS Cloud. This blog post will guide you through migrating your traditional SFTP server to AWS SFTP Transfer Family, focusing on a secure, scalable, and highly available architecture.
Why Migrate to AWS SFTP Transfer Family?
AWS SFTP Transfer Family provides a robust, serverless alternative to traditional SFTP servers. Here are some key benefits:
- Ease of Management: AWS handles the underlying infrastructure, reducing operational burden.
- High Availability: Native multi-AZ support ensures uninterrupted service.
- Scalability: Seamless integration with Amazon S3 allows for virtually unlimited storage capacity.
- Security: Built-in key-based authentication and IAM for granular control over user access.
- Cost-Effectiveness: Pay only for the resources you use, with no upfront investment in hardware.
- Customizable architecture design: AWS Cloud enables you to design your SFTP architecture based on specific customer requirements, utilizing network components such as VPCs, security groups, and NLBs to achieve the most secure and stringent configurations as needed.
- Seamless integration for microservices: Files available on S3 storage supported SFTP Transfer family allows for seamless integrations for microservice development on AWS.
Proposed Architecture
For this migration, I propose a secure and serverless architecture with the following components based on one of our Financial Services customer's on-premises to AWS Migration scenario:
Internal Setup with a Firewall: Use a firewall (e.g., Fortinet) in front of your SFTP server to secure and inspect incoming traffic.
Route traffic through a Network Load Balancer (NLB) configured for high availability.SFTP Server in a VPC: Deploy the SFTP server in a Virtual Private Cloud (VPC) for network isolation.
High Availability with 2 AZs: Configure the SFTP server to operate across two Availability Zones (AZs) for resilience.
Service-Managed Users: Use service-managed user accounts with key-based authentication for secure access.
S3 Backend Storage: Store files in Amazon S3 for scalability and durability, and leverage lifecycle policies to optimize storage costs.
Fine-Grained Access Control with IAM: Use IAM roles to enforce folder-level permissions for secure and organized access.
Implementation Steps
Following is the step by step implementation details.
Set Up AWS SFTP Transfer Family
a. Navigate to the AWS Transfer Family service in the AWS Management Console.
b. Create a new SFTP server and configure it as VPC Hosted and internal access with two (or three) Availability Zones.
c. Attach a security group that permits traffic only from the firewall.
d. Configure a Server host key by adding an already generated private SSH key, which will be presented when users access the sftp server.
Configure Network Load Balancer (NLB)
a. Deploy an NLB in front of the SFTP server.
b. Configure the NLB to route traffic to the SFTP endpoint on port 22.
c. Set up health checks for continuous monitoring.
Integrate the Firewall
a. Use a Fortinet (or similar) firewall to control, inspect and monitor incoming requests.
b. Allow only specific IP ranges or VPN traffic through the firewall to the NLB.
c. Whitelist sftp partner/customer IP addresses to restrict access to only required inbound connections
Set Up Service-Managed Users
a. Request SFTP partner/customer to provide the SSH Public key for secure connection access as a prerequisite step, or reuse the SSH public key from on-premises or current SFTP setup.
b. Define user accounts in AWS Transfer Family and assign each user a unique SSH public key.
c. Map each user to an S3 bucket or folder for isolated file access.
d. Setup Home directory for each user with the bucket or individual username folder or any directory structure you want to use, check 'Restricted' checkbox for the sftp user to not access anything outside this folder. The user will not be able to see the s3 bucket or folder name when Restricted is checked.
AWS SFTP Server summary page for reference:
Connect S3 as Backend Storage
a. Attach Amazon S3 as the backend storage for the SFTP server.
b. Configure lifecycle policies to transition data to lower-cost storage classes (e.g., S3 Glacier).
c. Configure S3 copy or backup to copy files to another s3 bucket if needed so you can directly process the files from the SFTP bucket from your applications.
Configure IAM Roles for Fine-Grained Access Control
a. Define IAM roles to control access to specific S3 folders with read/write/delete permissions.
b. You can use different IAM roles for different users if you want to provide additional access like 's3:DeleteObject' for certain folders within the home directory.
Example policy:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": ["s3:ListBucket"],
"Resource": "arn:aws:s3:::your-bucket",
"Condition": {
"StringLike": {
"s3:prefix": ["user-folder/*"]
}
}
},
{
"Effect": "Allow",
"Action": ["s3:GetObject", "s3:PutObject"],
"Resource": "arn:aws:s3:::your-bucket/user-folder/*"
}
]
}
The trust policy for the IAM role should be:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": "transfer.amazonaws.com"
},
"Action": "sts:AssumeRole",
"Condition": {}
}
]
}
Test and Validate
a. Enable CloudWatch Logs for logging and troubleshooting of SFTP connections.
b. Test user connections through the firewall and NLB. Ask your external sftp users to connect via any SFTP clients such as FileZilla or WinSCP. Note that the AWS SFTP server host doesn't allow SSH connection, you need sftp command to connect.
c. Validate user access permissions to S3 folders.
d. Simulate failover scenarios to confirm high availability.
Migration Plan
Use this simple migration plan for migrating from traditional/on-premises SFTP server to AWS SFTP Transfer Family
Inform SFTP Users About Changes
a. Notify all existing SFTP users about the migration to the new AWS SFTP setup.
b. Share details on timelines, new connection endpoints, and any required actions from their side.
Transition to Key-Based Authentication
a. Convert all users from password-based authentication to SSH key-based authentication, as AWS SFTP Transfer Family does not support password-based logins as it is not a secure way to access.
b. Assist users in generating and uploading their SSH public keys.
Onboard and Migrate Users
a. Create service-managed user accounts in AWS Transfer Family.
b. Migrate users' home directories and set up their specific access permissions in Amazon S3.
Set Up and Validate Access
a. Validate that all users can access their respective directories and files as expected.
b. Conduct testing to ensure smooth operations and troubleshoot any access issues.
Go Live
a. Update DNS or endpoint configurations to point to the new endpoint, in this case a public IP (setup DNS for this IP) on Fortinet Firewall.
b. Officially transition all users to the new setup and monitor for any post-migration issues.
Cost Optimization Tips
- Enable S3 lifecycle policies to automatically move infrequently accessed data to lower-cost storage classes.
- Monitor usage with AWS Cost Explorer and set up budgets for cost control.
- Use Savings Plans for additional savings.
Conclusion
Migrating from traditional SFTP servers to AWS SFTP Transfer Family offers significant advantages in terms of scalability, security, and cost efficiency. By leveraging the architecture and steps outlined in this guide, you can seamlessly transition to a serverless, fully managed solution that simplifies operations and improves reliability. Embrace AWS SFTP Transfer Family and future-proof your file transfer needs today.
Top comments (0)