Introduction

In this post, I'll summarize the provisioning of AWS resources with Terraform, the deployment of the Dockerized flask web-app to those resources and creating a CI/CD pipeline with GitHub actions.
NOTE: This is a summary of the project, to access the complete step by step process for the deployment of the project, click the link below

Link Available soon

Project Architecture

Here, the AWS services used for the project will be defined, alongside the AWS Architecture diagram

AWS Architecture Diagram

Terraform

All infrastructure resources for this project were provisioned using Terraform with a modular approach. Each component, from the VPC to ECS, was defined within its respective Terraform module for clarity and reusability.
The Terraform backend was created first, consisting of an S3 bucket for state storage and a DynamoDB table for state locking, ensuring safe concurrent operations. Then the remaining resources were provisioned next.

Backend Configuration

Before provisioning the resources, I set up the Terraform backend. This is very important as it is where terraform will store the state files, and it is important this is separate from the main infrastructure.
The following resources were deployed for the backend:

• S3 Bucket: Stores the Terraform state file.
• DynamoDB Table: Manages state locks to prevent concurrent changes.

This ensures safe, versioned state management for the infrastructure.

Deployment resources

The main Infrastructure is provisioned here.

1. Virtual Private Cloud (VPC) Module

VPC: I created a VPC in the us-east-1 region

Subnets: Following the VPC, I made 4 subnets in two availability zones:

• Public Subnets: I made two public subnets in the us-east 1a and us-east 1b availability zones for the internet-facing Application Load Balancer (ALB) and other resources that require internet.
• Private Subnets: I made two private subnets in the us-east 1a and us-east 1b availability zones as well but this time it's for my ECS service tasks. This makes a more secure architecture.

Internet Gateway (IGW): I made the IGW to give the VPC and public subnets internet access.

Route Tables: I created the two route tables

• Public Route-table: I made this route table to link the VPC to the Internet Gateway (IGW).
• Private Route-table: I made this route table to link my private subnets with VPC endpoints.

Route Table Associations: I created this in order to associate my subnets to their respective route tables. i.e. Private Subnet to private route table and public subnet to public route table.

VPC Endpoints: The VPC endpoints enable the ECS tasks in the private subnet to access certain resources.

• ECR and Docker endpoints: Ensure ECS tasks can pull Docker images.
• CloudWatch endpoint: For secure logging.
• S3 gateway endpoint: Access data and configurations securely.

2. Application Load Balancer

Listener: I created the listener to forward traffic to the ECS target group. listens on the port 80 (HTTP)
Target Group: The Target Group routes the traffic requests from the listener to exposed docker port in the ECS tasks in the private subnets.
Security group I created a security group for the ALB to specify what type of traffic to allow and on what port.

3. Elastic Container Service (ECS)

ECS Service: Here, the desired count of tasks is specified, the launch type, the subnets to be deployed in, the security groups, load balancer & target group, and the container port are all specified in this resource.
Task Definitions: Here, the task definition required by the ECS service is provisioned. The CPU, memory, execution role and container definitions are specified here.
IAM service roles and execution role: I created the necessary IAM service roles for the ECS service and the task execution role for the task definition.

4. Elastic Container Registry (ECR)

• I Created this resource to hold my docker image.

6. Route 53

• I used this service for DNS configuration for the application domain, routing traffic to the ALB.

Challenges faced and solutions

1.ECS tasks not being able to access the ALB
Solution:
This was due to networking misconfigurations with security groups and routing. I adjusted the security group settings to ensure proper communication between the ECS tasks and the ALB, including allowing inbound HTTP traffic on port:8080. Additionally, I verified that the ALB was correctly configured to route traffic to the ECS task group.

1. S3 bucket state storage and DynamoDB locking conflicts Solution: When setting up the backend for Terraform, I encountered issues with the terraform destroy -auto-approve command due to the S3 bucket and DynamoDB table. These resources were held my terraform state files and were defined in the main infrastructure, when I tried to delete also deleted my bucket and table, which contained my state files, causing issues when I want to provision the main infrastructure again. Solution To resolve this, I separated the Terraforms backend and min infrastructure.

Future Improvements

I plan to integrate unit tests in the pipeline.

Thank you for reading, check out my profile, for more Cloud and DevOps posts just like this

Relevant Links

• Checkout the project on my GitHub

  VSI12 / Terraform-ECS-IDS

  Deploying a Containerized Web-App to AWS ECS Using Terraform and CI/CD

  Project Overview

  This project focuses on deploying a dockerized Flask Classification based Intrusion Detection System (IDS) to AWS ECS (Elastic Container Service) using Terraform for provisioning AWS infrastructure and GitHub Actions for CI/CD automation The IDS allows users to upload network traffic datasets (formatted like the NSL-KDD dataset), analyze them for potential threats, and visualize the results.

  The deployment architecture leverages AWS services such as Virtual Private Cloud (VPC) ECS (with Fargate), ECR (Elastic Container Registry), an Application Load Balancer (ALB), and VPC endpoints for secure network communication. The entire infrastructure is managed as code with Terraform, ensuring consistency, scalability, and easy maintenance.

  Architecture

  • Virtual Private Cloud (VPC): Configured with public and private subnets across two availability zones for high availability and security.
  • Interget Gateway: Enables communication between the VPC and the internet
  • VPC Endpoints: The…

  View on GitHub