EduardoCusihuaman

Posted on Aug 19

Containers, Containers, Containers!

#aws #github #cicd #containers

Yep, you already know how to build your container. But now you want to get that container running on AWS with as little fuss as possible, right? No problem. ⚡

In this quick guide, I'll show you how to deploy your container to AWS Fargate using CloudFormation. It’s fast, easy, and doesn’t require you to wade through endless steps. Just the essentials to get your container online.

Before We Begin

We’re using CloudFormation here because it’s perfect for quickly spinning up your infrastructure with minimal hassle. Sure, there are other tools like Terraform or CDK, but for a straightforward ECS setup, CloudFormation is your go-to.

We’re keeping it simple: we’ll create a VPC, a couple of public subnets, an ECS cluster, and a load balancer to get your containers up and running. No need for extra complexity—just the basics to get your containerized app deployed fast. And yes, we’ve got IAM roles to keep everything secure. Let’s dive in!

Architecture Boring Stuff
LET'S KICK IT OFF!
Pricing
Conclusion

This table of contents should make it easy to navigate through the guide.

Architecture Boring Stuff

User Interaction

User Request: A user sends an HTTP request to the Application Load Balancer (ALB) (ALBDNSName).
ALB Routing: The ALB routes the request to the ECS service, which runs your containerized application.
ECS Task Handling: The ECS tasks, running within the Fargate cluster, handle the request, with containers managed by ECS.

ECS and Load Balancer

ECS Fargate Cluster (ECSFargateCluster): Manages and scales the containers based on defined parameters.
Application Load Balancer (ALB): Distributes incoming traffic to the appropriate ECS tasks, ensuring availability and responsiveness.
Target Group (ALBTargetGroup): Monitors the health of your ECS tasks and directs traffic only to healthy instances.

IAM Roles

IAMRoleForECS: Grants permissions for ECS tasks to pull images from ECR, send logs to CloudWatch, and perform other necessary actions.
ECRCleanupLambdaRole: Allows the Lambda function to delete images from the ECR repository during the cleanup process.

Pipeline User for GitHub Actions

The GitHub Actions workflow uses an IAM user (PipelineUser) to deploy updates to the ECS service. The workflow includes steps for:

Checking out the code.
Building the container image.
Pushing the image to ECR.
Configuring AWS credentials.
Updating the ECS task definition with the new image.
Triggering a new deployment of the ECS service.

Flow

User -> Application Load Balancer -> ECS Service -> ECS Task (Container)
ECS Tasks -> Fargate Cluster (for running containers)
Pipeline User -> GitHub Actions -> ECR and ECS (for deployment)

cloudformation-template.yaml

AWSTemplateFormatVersion: "2010-09-09"
Description: AWS ECS Infrastructure setup using a new VPC

Parameters:
    VpcCidr:
        Description: "CIDR block for the VPC"
        Type: String
        Default: "30.0.0.0/16"

    PublicSubnet1Cidr:
        Description: "CIDR block for the public subnet 1"
        Type: String
        Default: "30.0.1.0/24"

    PublicSubnet2Cidr:
        Description: "CIDR block for the public subnet 2"
        Type: String
        Default: "30.0.2.0/24"

    ECRRepositoryName:
        Description: The name of the ECR repository
        Type: String
        Default: "application-repository"

    ClusterName:
        Description: The name of the ECS Fargate Cluster
        Type: String
        Default: "EcsFargateCluster"

    ServiceName:
        Description: The name of the ECS Service
        Type: String
        Default: "EcsService"

    TaskFamilyName:
        Description: The family name of the ECS Task Definition
        Type: String
        Default: "TaskDefinition"

    ContainerName:
        Description: The name of the container in the ECS Task Definition
        Type: String
        Default: "application-container"

Resources:
    # VPC and Networking resources
    VPC:
        Type: AWS::EC2::VPC
        Properties:
            CidrBlock: !Ref VpcCidr
            EnableDnsSupport: true
            EnableDnsHostnames: true
            Tags:
                - Key: Name
                  Value: vpc

    # Internet Gateway
    InternetGateway:
        Type: AWS::EC2::InternetGateway
        Properties:
            Tags:
                - Key: Name
                  Value: igw

    AttachGateway:
        Type: AWS::EC2::VPCGatewayAttachment
        Properties:
            VpcId: !Ref VPC
            InternetGatewayId: !Ref InternetGateway

    # Public Subnets
    PublicSubnet1:
        Type: AWS::EC2::Subnet
        Properties:
            VpcId: !Ref VPC
            CidrBlock: !Ref PublicSubnet1Cidr
            AvailabilityZone: !Select
                - 0
                - Fn::GetAZs: !Ref "AWS::Region"
            MapPublicIpOnLaunch: true
            Tags:
                - Key: Name
                  Value: public-subnet-1

    PublicSubnet2:
        Type: AWS::EC2::Subnet
        Properties:
            VpcId: !Ref VPC
            CidrBlock: !Ref PublicSubnet2Cidr
            AvailabilityZone: !Select
                - 1
                - Fn::GetAZs: !Ref "AWS::Region"
            MapPublicIpOnLaunch: true
            Tags:
                - Key: Name
                  Value: public-subnet-2

    # Route Tables and Routes
    PublicRouteTable:
        Type: AWS::EC2::RouteTable
        Properties:
            VpcId: !Ref VPC
            Tags:
                - Key: Name
                  Value: public-rt

    PublicRoute:
        Type: AWS::EC2::Route
        Properties:
            RouteTableId: !Ref PublicRouteTable
            DestinationCidrBlock: 0.0.0.0/0
            GatewayId: !Ref InternetGateway

    PublicSubnet1RouteTableAssociation:
        Type: AWS::EC2::SubnetRouteTableAssociation
        Properties:
            SubnetId: !Ref PublicSubnet1
            RouteTableId: !Ref PublicRouteTable

    PublicSubnet2RouteTableAssociation:
        Type: AWS::EC2::SubnetRouteTableAssociation
        Properties:
            SubnetId: !Ref PublicSubnet2
            RouteTableId: !Ref PublicRouteTable

    # Security Group
    SecurityGroup:
        Type: AWS::EC2::SecurityGroup
        Properties:
            GroupDescription: "Default security group for the VPC"
            VpcId: !Ref VPC
            SecurityGroupIngress:
                - IpProtocol: tcp
                  FromPort: 80
                  ToPort: 80
                  CidrIp: 0.0.0.0/0
                - IpProtocol: tcp
                  FromPort: 443
                  ToPort: 443
                  CidrIp: 0.0.0.0/0
                - IpProtocol: tcp
                  FromPort: 22
                  ToPort: 22
                  CidrIp: 0.0.0.0/0
            SecurityGroupEgress:
                - IpProtocol: "-1"
                  CidrIp: 0.0.0.0/0
            Tags:
                - Key: Name
                  Value: default-sg

    # ECS Fargate Cluster
    ECSFargateCluster:
        Type: AWS::ECS::Cluster
        Properties:
            ClusterName: !Ref ClusterName

    # ECR Repository
    ECRRepository:
        Type: AWS::ECR::Repository
        Properties:
            RepositoryName: !Ref ECRRepositoryName

    # Custom Resource Hook for ECR Cleanup
    ECRCleanupHook:
        Type: "Custom::ECRCleanup"
        Properties:
            ServiceToken: !GetAtt ECRCleanupLambdaFunction.Arn
            RepositoryName: !Ref ECRRepositoryName
        DependsOn: ECRRepository

    # ECS Task Definition
    ECSTaskDefinition:
        Type: AWS::ECS::TaskDefinition
        Properties:
            Family: !Ref TaskFamilyName
            NetworkMode: awsvpc
            RequiresCompatibilities:
                - FARGATE
            Cpu: "512"
            Memory: "1024"
            ExecutionRoleArn: !Ref IAMRoleForECS
            TaskRoleArn: !Ref IAMRoleForECS
            ContainerDefinitions:
                - Name: !Ref ContainerName
                  Image: "nginx:latest"
                  PortMappings:
                      - ContainerPort: 80
                  Environment:
                      - Name: ENVIRONMENT
                        Value: "production"
                  LogConfiguration:
                      LogDriver: awslogs
                      Options:
                          awslogs-group: !Ref CloudWatchLogGroup
                          awslogs-region: !Ref "AWS::Region"
                          awslogs-stream-prefix: ecs

    # ECS Service
    ECSService:
        Type: AWS::ECS::Service
        DependsOn: ALB
        Properties:
            Cluster: !Ref ECSFargateCluster
            ServiceName: !Ref ServiceName
            DesiredCount: 2
            LaunchType: FARGATE
            TaskDefinition: !Ref ECSTaskDefinition
            NetworkConfiguration:
                AwsvpcConfiguration:
                    AssignPublicIp: ENABLED
                    SecurityGroups:
                        - !Ref SecurityGroup
                    Subnets:
                        - !Ref PublicSubnet1
                        - !Ref PublicSubnet2
            LoadBalancers:
                - ContainerName: !Ref ContainerName
                  ContainerPort: 80
                  TargetGroupArn: !Ref ALBTargetGroup

    # ALB
    ALB:
        Type: AWS::ElasticLoadBalancingV2::LoadBalancer
        Properties:
            Name: "ALB"
            Subnets:
                - !Ref PublicSubnet1
                - !Ref PublicSubnet2
            SecurityGroups:
                - !Ref SecurityGroup
            LoadBalancerAttributes:
                - Key: idle_timeout.timeout_seconds
                  Value: "60"

    ALBListener:
        Type: AWS::ElasticLoadBalancingV2::Listener
        Properties:
            LoadBalancerArn: !Ref ALB
            Port: 80
            Protocol: HTTP
            DefaultActions:
                - Type: forward
                  TargetGroupArn: !Ref ALBTargetGroup

    ALBTargetGroup:
        Type: AWS::ElasticLoadBalancingV2::TargetGroup
        Properties:
            VpcId: !Ref VPC
            Protocol: HTTP
            Port: 80
            TargetType: ip
            HealthCheckEnabled: true
            HealthCheckPath: "/" # Set to a valid endpoint in your application
            HealthCheckIntervalSeconds: 30
            HealthCheckTimeoutSeconds: 5
            HealthyThresholdCount: 2
            UnhealthyThresholdCount: 3
            Matcher:
                HttpCode: "200"
            Name: "TargetGroup"

    # IAM Role for ECS Task
    IAMRoleForECS:
        Type: AWS::IAM::Role
        Properties:
            AssumeRolePolicyDocument:
                Version: "2012-10-17"
                Statement:
                    - Effect: Allow
                      Principal:
                          Service:
                              - ecs-tasks.amazonaws.com
                      Action:
                          - sts:AssumeRole
            Policies:
                - PolicyName: "ECRPullPolicy"
                  PolicyDocument:
                      Version: "2012-10-17"
                      Statement:
                          - Effect: Allow
                            Action:
                                - ecr:GetAuthorizationToken
                                - ecr:GetDownloadUrlForLayer
                                - ecr:BatchGetImage
                                - ecr:BatchCheckLayerAvailability
                            Resource: "*"
                - PolicyName: "CloudWatchLogsPolicy"
                  PolicyDocument:
                      Version: "2012-10-17"
                      Statement:
                          - Effect: Allow
                            Action:
                                - logs:CreateLogStream
                                - logs:PutLogEvents
                                - logs:CreateLogGroup
                            Resource: "*"

    # CloudWatch Log Group
    CloudWatchLogGroup:
        Type: AWS::Logs::LogGroup
        Properties:
            LogGroupName: "/ecs/logs"
            RetentionInDays: 7

    # Lambda Function to Delete ECR Images
    ECRCleanupLambdaRole:
        Type: AWS::IAM::Role
        Properties:
            AssumeRolePolicyDocument:
                Version: "2012-10-17"
                Statement:
                    - Effect: Allow
                      Principal:
                          Service: lambda.amazonaws.com
                      Action: sts:AssumeRole
            Policies:
                - PolicyName: LambdaCloudWatchLogsPolicy
                  PolicyDocument:
                      Version: "2012-10-17"
                      Statement:
                          - Effect: Allow
                            Action:
                                - logs:CreateLogGroup
                                - logs:CreateLogStream
                                - logs:PutLogEvents
                            Resource:
                                - arn:aws:logs:*:*:*
                - PolicyName: ECRDeletePolicy
                  PolicyDocument:
                      Version: "2012-10-17"
                      Statement:
                          - Effect: Allow
                            Action:
                                - ecr:BatchDeleteImage
                                - ecr:ListImages
                            Resource: !Sub "arn:aws:ecr:${AWS::Region}:${AWS::AccountId}:repository/${ECRRepositoryName}"

    ECRCleanupLambdaFunction:
        Type: AWS::Lambda::Function
        Properties:
            FunctionName: ecr-cleanup-lambda
            Handler: index.handler
            Runtime: python3.9
            Role: !GetAtt ECRCleanupLambdaRole.Arn
            Timeout: 60
            Code:
                ZipFile: |
                    import boto3
                    def handler(event, context):
                        if event['RequestType'] == 'Delete':
                            ecr = boto3.client('ecr')
                            images_to_delete = ecr.list_images(repositoryName=event['repository_name'])['imageIds']
                            if images_to_delete:
                                ecr.batch_delete_image(
                                    repositoryName=event['repository_name'],
                                    imageIds=images_to_delete
                                )
                        return 'Cleanup complete'

Outputs:
    VpcId:
        Value: !Ref VPC
        Description: The VPC Id where the ECS Cluster is deployed.
    ECSFargateCluster:
        Value: !Ref ECSFargateCluster
        Description: The ECS Fargate Cluster created by the template.
    ALBDNSName:
        Value: !GetAtt ALB.DNSName
        Description: The DNS name of the Application Load Balancer.
    ALBEndpoint:
        Value: !Sub "http://${ALB.DNSName}"
        Description: The HTTP endpoint of the Application Load Balancer.

The same template without creating a new vpc:
cloudformation-template-with-existing-vpc.yaml

Note: It's important to edit the task's port if needed. By default, this is set up with a placeholder using nginx listening on port 80 and the health check is set to /. However, you can change these settings to fit your requirements.

LET'S KICK IT OFF!

Alright, buckle up! If you've got an AWS account and the AWS CLI locked and loaded, we're ready to roll. (Pro tip: jq is handy, but hey, we won't judge if you skip it.)

Step-01: Deploy CloudFormation Stack for ECS Fargate

export CLUSTER_NAME=EcsFargateCluster
export SERVICE_NAME=EcsService
export TASK_FAMILY_NAME=TaskDefinition
export REPOSITORY_NAME=application-repository
export CONTAINER_NAME=application-container
export STACK_NAME=aws-ecs-fargate-stack

aws cloudformation deploy \
    --stack-name $STACK_NAME \
    --template-file ecs-fargate-cloudformation.yaml \
    --parameter-overrides \
        ClusterName=$CLUSTER_NAME \
        ServiceName=$SERVICE_NAME \
        TaskFamilyName=$TASK_FAMILY_NAME \
        ECRRepositoryName=$REPOSITORY_NAME \
        ContainerName=$CONTAINER_NAME \
    --capabilities CAPABILITY_NAMED_IAM \
    --region us-east-1

Note: The stack uses predefined default names for easy identification in the pipeline, u can override it:

ClusterName: EcsFargateCluster
ServiceName: EcsService
TaskFamilyName: TaskDefinition
ECRRepositoryName: application-repository
ContainerName: application-container

Step-02: Create AWS IAM User for Pipeline

In this step, we'll create a Pipeline User that GitHub Actions will use. This user will have the necessary policies attached to push container images to ECR and update our ECS service. Here's how to do it:

# Set user name
export USER_NAME="codepipeline"

# Create IAM user
aws iam create-user --user-name $USER_NAME

# Attach necessary policies
# ⚠️ THIS IS NOT PRODUCTION READY - USE A LEAST PRIVILEGE ROLE INSTEAD
aws iam attach-user-policy --user-name $USER_NAME --policy-arn arn:aws:iam::aws:policy/AmazonECS_FullAccess
aws iam attach-user-policy --user-name $USER_NAME --policy-arn arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryFullAccess

# Create access keys for the user
ACCESS_KEYS=$(aws iam create-access-key --user-name $USER_NAME)

# Extract AccessKeyId and SecretAccessKey
ACCESS_KEY_ID=$(echo $ACCESS_KEYS | jq -r '.AccessKey.AccessKeyId')
SECRET_ACCESS_KEY=$(echo $ACCESS_KEYS | jq -r '.AccessKey.SecretAccessKey')

# ⚠️ Output the Access Key details
echo "Access Key ID: $ACCESS_KEY_ID"
echo "Secret Access Key: $SECRET_ACCESS_KEY"

Step-03: Add Secrets to GitHub Secrets

After creating the IAM user and generating the access keys, follow these steps to add these credentials to your GitHub
repository secrets for use in GitHub Actions:

Navigate to Your GitHub Repository:
- Go to the main page of your repository on GitHub.
Access Repository Settings:
- Click on the Settings tab at the top of the repository page.
Navigate to Secrets:
- In the left sidebar, click on Secrets and variables > Actions.
Add New Repository Secrets:
- Click New repository secret and add the following:

AWS_ACCESS_KEY_ID: Enter your AccessKeyId value.
AWS_SECRET_ACCESS_KEY: Enter your SecretAccessKey value.

By doing this, you'll securely store the necessary credentials for your GitHub Actions workflow.

Step-04: Github Actions

Time to automate the deployment with GitHub Actions! Here's how you can set up your pipeline:

Copy the following YAML content into .github/workflows/deploy.yml:

name: Build, Push to ECR, and Deploy to ECS Fargate

on:
  push:
    branches:
      - main

jobs:
  build-and-deploy:
    runs-on: ubuntu-latest

    steps:
      # Step 1: Checkout the latest code from the main branch
      - name: Checkout code
        uses: actions/checkout@v3

      # Step 2: Configure AWS credentials for the GitHub Actions runner
      - name: Configure AWS credentials
        uses: aws-actions/configure-aws-credentials@v1
        with:
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          aws-region: us-east-1 # Change to your region

      # Step 3: Login to Amazon ECR to push Docker images
      - name: Login to Amazon ECR
        id: login-ecr
        uses: aws-actions/amazon-ecr-login@v1

      # Step 4: Build the Docker image, tag it, and push it to Amazon ECR
      - name: Build, tag, and push image to Amazon ECR
        id: build-image
        env:
          ECR_REGISTRY: ${{ steps.login-ecr.outputs.registry }}
          ECR_REPOSITORY: application-repository # Change this to your ECR repository name
          IMAGE_TAG: ${{ github.sha }}
        run: |
          docker build -t $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG .
          docker push $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG
          echo "image=$ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG" >> $GITHUB_OUTPUT

      # Step 5: Download the current ECS task definition to update it with the new image
      - name: Download current task definition
        run: |
          aws ecs describe-task-definition --task-definition TaskDefinition \ # Change this to your Task Family Name
          --query taskDefinition > ecs-task-definition.json

      # Step 6: Replace the image in the ECS task definition with the newly built image
      - name: Fill in the new image ID in the Amazon ECS task definition
        id: task-def
        uses: aws-actions/amazon-ecs-render-task-definition@v1
        with:
          task-definition: ecs-task-definition.json
          container-name: application-container # Change this to your container name
          image: ${{ steps.build-image.outputs.image }}

      # Step 7: Deploy the updated task definition to the ECS Fargate service
      - name: Deploy updated task definition to ECS Fargate
        uses: aws-actions/amazon-ecs-deploy-task-definition@v2
        with:
          task-definition: ${{ steps.task-def.outputs.task-definition }}
          service: EcsService # Change this to your ECS service name
          cluster: EcsFargateCluster # Change this to your ECS Fargate cluster name
          wait-for-service-stability: true

Important: The container image in the ECR repository is updated with each push to the main branch. The service name EcsService, cluster name EcsFargateCluster, and other values are predefined in your CloudFormation stack, ensuring smooth integration with your AWS infrastructure.

Now, commit and push your changes to trigger the pipeline:

git commit -am “feat(ci): deployment”
git push origin main

Once pushed, the pipeline will automatically build, push your Docker image to ECR, and update your ECS Fargate service with the latest code. 🚀

Cleaning Up

To delete the CloudFormation stack, use the following command:

# Set the stack name
export STACK_NAME="aws-ecs-fargate-stack"

# Delete the CloudFormation stack
aws cloudformation delete-stack --stack-name $STACK_NAME

# Delete the IAM user
aws iam delete-user --user-name $USER_NAME

BLAZINGGG ENJOYY 🎉🔥

You can see your pipeline triggered, the Docker image built, pushed to ECR, and your ECS Fargate service updated. This workflow will automatically deploy your changes to the Fargate cluster every time you push to the main branch. 🚀

To check if everything's working, grab the ALB DNS Name from the stack output or the AWS console and make an HTTP request to your application. 🌐

~/edu curl http://your-alb-dns-name/ # Replace with your actual ALB DNS name

Pricing

Here’s an estimated cost breakdown for the AWS ECS infrastructure setup you provided, considering that you have a single ECS Fargate task running:

1. VPC and Subnets

VPC: No additional cost.
Subnets: No direct cost for subnets.

2. Internet Gateway

Cost: $0.045 per hour.

3. ECS Fargate

vCPU (0.5 vCPU): $0.04048 per vCPU-hour.
Memory (1 GB): $0.004445 per GB-hour.

For a single task running 24/7:

vCPU Cost (0.5 vCPU): 0.5 * 24 * 30 * $0.04048 = $14.58 per month.
Memory Cost (1 GB): 1 * 24 * 30 * $0.004445 = $3.20 per month.

4. Application Load Balancer (ALB)

ALB: $0.0225 per hour.
Data Processed by ALB: $0.008 per GB.

For 24/7 operation:

ALB Cost: 24 * 30 * $0.0225 = $16.20 per month.

5. ECR Storage

Cost: $0.10 per GB per month.

6. CloudWatch Logs

Storage: $0.03 per GB.
Ingested Logs: $0.50 per GB.

Total Estimated Monthly Cost

Adding up the individual components, here’s an estimated monthly cost:

ECS Fargate: ~$17.78
ALB: ~$16.20
Internet Gateway: ~$32.40 (if used)
ECR Storage: Variable depending on image size.
CloudWatch Logs: Variable depending on log volume.

Estimated Total: ~$66.38 per month (excluding data transfer and storage variations).

Alternative Pricing without Internet Gateway:
If you use the default AWS network configuration and don’t need an Internet Gateway:

Estimated Total: ~$34.58 per month (excluding Internet Gateway cost).

Conclusion

While serverless and Fargate offer great convenience, costs can escalate quickly. For instance, running a 0.5 vCPU and 1GB memory task costs around $18/month just for compute. Additional services like the Application Load Balancer (ALB) and CloudWatch Logs further increase expenses. If you're working on smaller projects or development environments, consider skipping features like the Internet Gateway or ALB to save costs. Always tailor your setup to your specific needs to optimize cloud spending.

And there you have it! A blazing fast, simple way to deploy your containers on AWS ECS Fargate. Enjoy!

Happy deploying :)

Top comments (1)

Luis • Aug 24

🐳🐋

DEV Community