DEV Community

Cover image for CI/CD Pipeline for Amazon ECS/FARGATE with Terraform
erozedguy
erozedguy

Posted on • Updated on

CI/CD Pipeline for Amazon ECS/FARGATE with Terraform

DESCRIPTION

In this post I am going to explain how to build the infrastructure on AWS with Terraform to implement a CI / CD pipeline for ECS / Fargate.
The Architecture consists of a VPC with 2 public subnets in different Availability Zones. The desired tasks are 2 and each task is deployed on each public subnet with Fargate and each task belongs to the same ECS service.
An Application Load Balancer is used to balance the load between the two tasks.
In this case, the main goal is to implement a docker container that contains a simple HTTP server built with GOLANG. This HTTP server allows you to obtain the private IP of each task.
If I push new changes to the CodeCommit repository, CodePipeline detects those changes, triggers the pipeline and creates a new docker image and then deploys it to the ECS service to update the tasks.

ARCHITECTURE

aws cicd pipeline

RESOURCES

https://github.com/erozedguy/CICD-Pipeline-for-Amazon-ECS-Fargate

STEPS

STEP 01 - Create a IAM Role and CodeCommit Credentials

  • Create a Service Role for Elastic Container Service Task (
    Allows ECS tasks to call AWS services on your behalf.)
    taskRole

  • Attach AWSCodeCommitPowerUser Policy to my USER
    commitpolicy

  • Generate HTTPS Git credentials for AWS CodeCommit to clone, push, pull to the CodeCommit Repository

credentials

STEP 02: Terraform scripts to build the infrastructure

PROVIDERS

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 3.51"
    }
  }
}
provider "aws" {
  profile = "default"
  region  = "us-east-1"
}
Enter fullscreen mode Exit fullscreen mode

VPC

The vpc script has VPC, SUBNETS and INTERNET GATEWAY resources.

resource "aws_vpc" "ecs-vpc" {
  cidr_block = "${var.cidr}"

  tags = {
    Name = "ecs-vpc"
  }
}

# PUBLIC SUBNETS
resource "aws_subnet" "pub-subnets" {
  count                   = length(var.azs)
  vpc_id                  = "${aws_vpc.ecs-vpc.id}"
  availability_zone       = "${var.azs[count.index]}"
  cidr_block              = "${var.subnets-ip[count.index]}"
  map_public_ip_on_launch = true

  tags = {
    Name = "pub-subnets"
  }
}

# INTERNET GATEWAY
resource "aws_internet_gateway" "i-gateway" {
  vpc_id = "${aws_vpc.ecs-vpc.id}"

  tags = {
    Name = "ecs-igtw"
  }
}
Enter fullscreen mode Exit fullscreen mode

VARIABLES TO VPC

variable "cidr" {
  type    = string
  default = "145.0.0.0/16"
}

variable "azs" {
  type = list(string)
  default = [
    "us-east-1a",
    "us-east-1b"
  ]
}

variable "subnets-ip" {
  type = list(string)
  default = [
    "145.0.1.0/24",
    "145.0.2.0/24"
  ]

}
Enter fullscreen mode Exit fullscreen mode

IAM ROLES & POLICIES

For the CodeBuild is necessary to create a IAM Role&Policy to allow access to ECR to push and pull the Docker images in the ECR repository. Also, is necessary a permission to access a S3 bucket to store the artifacts.

resource "aws_iam_role" "codebuild-role" {
  name = "codebuild-role"
  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Action = "sts:AssumeRole"
        Effect = "Allow"
        Principal = {
          Service = "codebuild.amazonaws.com"
        }
      },
    ]
  })
}

resource "aws_iam_role_policy" "codebuild-policy" {
  role = "${aws_iam_role.codebuild-role.name}"

  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Action   = ["codecommit:GitPull"]
        Effect   = "Allow"
        Resource = "*"
      },
      {
        Action = [
          "ecr:BatchCheckLayerAvailability",
          "ecr:GetDownloadUrlForLayer",
          "ecr:BatchGetImage",
          "ecr:CompleteLayerUpload",
          "ecr:GetAuthorizationToken",
          "ecr:InitiateLayerUpload",
          "ecr:PutImage",
        "ecr:UploadLayerPart"]
        Effect   = "Allow"
        Resource = "*"
      },
      {
        Action = [
          "logs:CreateLogGroup",
          "logs:CreateLogStream",
        "logs:PutLogEvents"]
        Effect   = "Allow"
        Resource = "*"
      },
      {
        Action = [
          "s3:PutObject",
          "s3:GetObject",
          "s3:GetObjectVersion",
          "s3:GetBucketAcl",
        "s3:GetBucketLocation"]
        Effect   = "Allow"
        Resource = "*"
      }
    ]


  })

}
Enter fullscreen mode Exit fullscreen mode

ROUTE TABLES

A single Route Table for both Public Subnets

# TABLE FOR PUBLIC SUBNETS
resource "aws_route_table" "pub-table" {
  vpc_id = "${aws_vpc.ecs-vpc.id}"
}

resource "aws_route" "pub-route" {
  route_table_id         = "${aws_route_table.pub-table.id}"
  destination_cidr_block = "0.0.0.0/0"
  gateway_id             = "${aws_internet_gateway.i-gateway.id}"
}

resource "aws_route_table_association" "as-pub" {
  count          = length(var.azs)
  route_table_id = "${aws_route_table.pub-table.id}"
  subnet_id      = "${aws_subnet.pub-subnets[count.index].id}"
}

Enter fullscreen mode Exit fullscreen mode

SECURITY GROUPS

The first Sec-Group is for the ECS Service

resource "aws_security_group" "sg1" {
  name        = "golang-server"
  description = "Port 5000"
  vpc_id      = aws_vpc.ecs-vpc.id

  ingress {
    description      = "Allow Port 5000"
    from_port        = 5000
    to_port          = 5000
    protocol         = "tcp"
    cidr_blocks      = ["0.0.0.0/0"]
    ipv6_cidr_blocks = ["::/0"]
  }

  egress {
    description = "Allow all ip and ports outboun"
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }
}

Enter fullscreen mode Exit fullscreen mode

The second Sec-Group is for the Application Load Balancer

resource "aws_security_group" "sg2" {
  name        = "golang-server-alb"
  description = "Port 80"
  vpc_id      = aws_vpc.ecs-vpc.id

  ingress {
    description      = "Allow Port 80"
    from_port        = 80
    to_port          = 80
    protocol         = "tcp"
    cidr_blocks      = ["0.0.0.0/0"]
    ipv6_cidr_blocks = ["::/0"]
  }

  egress {
    description = "Allow all ip and ports outboun"
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }
}
Enter fullscreen mode Exit fullscreen mode

APPLICATION LOAD BALANCER

resource "aws_lb" "app-lb" {
  name               = "app-lb"
  internal           = false
  load_balancer_type = "application"
  security_groups    = [aws_security_group.sg2.id]
  subnets            = ["${aws_subnet.pub-subnets[0].id}", "${aws_subnet.pub-subnets[1].id}"]

}
Enter fullscreen mode Exit fullscreen mode

Port #5000 is used in the Target Group because that port is used for the container

resource "aws_lb_target_group" "tg-group" {
  name        = "tg-group"
  port        = "5000"
  protocol    = "HTTP"
  vpc_id      = "${aws_vpc.ecs-vpc.id}"
  target_type = "ip"

}
Enter fullscreen mode Exit fullscreen mode

Port #80 is used for the Listener

resource "aws_lb_listener" "lb-listener" {
  load_balancer_arn = "${aws_lb.app-lb.arn}"
  port              = "80"
  protocol          = "HTTP"

  default_action {
    type             = "forward"
    target_group_arn = "${aws_lb_target_group.tg-group.arn}"
  }
}
Enter fullscreen mode Exit fullscreen mode

ECS & ECR

ECR repository
resource "aws_ecr_repository" "ecr-repo" {
  name = "ecr-repo"
}
Enter fullscreen mode Exit fullscreen mode
ECS Cluster
resource "aws_ecs_cluster" "ecs-cluster" {
  name = "clusterDev"
}
Enter fullscreen mode Exit fullscreen mode
Task Definition
  • In this part is important to specify the containerPort
  • Create a ENV VAR: export TF_VAR_uri_repo = <ID_ACCOUNT>.dkr.ecr.<REGION>.amazonaws.com/<ECR_REPOSITORY_NAME>
resource "aws_ecs_task_definition" "task" {
  family                   = "HTTPserver"
  network_mode             = "awsvpc"
  requires_compatibilities = ["FARGATE"]
  cpu                      = 256
  memory                   = 512
  execution_role_arn       = data.aws_iam_role.ecs-task.arn

  container_definitions = jsonencode([
    {
      name   = "golang-container"
      image  = "${var.uri_repo}:latest" #URI
      cpu    = 256
      memory = 512
      portMappings = [
        {
          containerPort = 5000
        }
      ]
    }
  ])
}
Enter fullscreen mode Exit fullscreen mode
ECS Service

Specify the load balancer block

resource "aws_ecs_service" "svc" {
  name            = "golang-Service"
  cluster         = "${aws_ecs_cluster.ecs-cluster.id}"
  task_definition = "${aws_ecs_task_definition.task.id}"
  desired_count   = 2
  launch_type     = "FARGATE"


  network_configuration {
    subnets          = ["${aws_subnet.pub-subnets[0].id}", "${aws_subnet.pub-subnets[1].id}"]
    security_groups  = ["${aws_security_group.sg1.id}"]
    assign_public_ip = true
  }

  load_balancer {
    target_group_arn = "${aws_lb_target_group.tg-group.arn}"
    container_name   = "golang-container"
    container_port   = "5000"
  }
}
Enter fullscreen mode Exit fullscreen mode

CI/CD PIPELINE

CodeCommit Repository
resource "aws_codecommit_repository" "repo" {
  repository_name = var.repo_name
}
Enter fullscreen mode Exit fullscreen mode
CodeBuild Project
resource "aws_codebuild_project" "repo-project" {
  name         = "${var.build_project}"
  service_role = "${aws_iam_role.codebuild-role.arn}"

  artifacts {
    type = "NO_ARTIFACTS"
  }

  source {
    type     = "CODECOMMIT"
    location = "${aws_codecommit_repository.repo.clone_url_http}"
  }

  environment {
    compute_type    = "BUILD_GENERAL1_SMALL"
    image           = "aws/codebuild/standard:5.0"
    type            = "LINUX_CONTAINER"
    privileged_mode = true
  }
}
Enter fullscreen mode Exit fullscreen mode
buildspec.yml
  • This file is very important to create the Docker Image and Pull it to ECR repository
  • To update the ECS service is important to specify the containerName and imageUri in a JSON file with the name imagedefinitions.json. This file is an artifact
  • This file must be in the CodeCommit repository
version: 0.2

phases:
  pre_build:
    commands:
      - echo Logging in to Amazon ECR...
      - echo $AWS_DEFAULT_REGION
      - aws ecr get-login-password --region $AWS_DEFAULT_REGION | docker login --username AWS --password-stdin 940401905947.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com
      - REPOSITORY_NAME="ecr-repo"      
      - REPOSITORY_URI=940401905947.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com/$REPOSITORY_NAME
      - COMMIT_HASH=$(echo $CODEBUILD_RESOLVED_SOURCE_VERSION | cut -c 1-7)
      - IMAGE_TAG=${COMMIT_HASH:=latest}
  build:
    commands:
      - echo Building the Docker image...
      - docker build -t $REPOSITORY_NAME:latest .
      - docker tag $REPOSITORY_NAME:latest $REPOSITORY_URI:latest
      - docker tag $REPOSITORY_NAME:latest $REPOSITORY_URI:$IMAGE_TAG
  post_build:
    commands:
      - docker push $REPOSITORY_URI:latest
      - docker push $REPOSITORY_URI:$IMAGE_TAG
      - printf '[{"name":"golang-container","imageUri":"%s"}]' $REPOSITORY_URI:$IMAGE_TAG > imagedefinitions.json

artifacts:
  files: imagedefinitions.json
Enter fullscreen mode Exit fullscreen mode
S3 Bucket to store the artifacts
resource "aws_s3_bucket" "bucket-artifact" {
  bucket = "eroz-artifactory-bucket"
  acl    = "private"
}
Enter fullscreen mode Exit fullscreen mode
CodePipeline

Specify Source, Build, Deploy Stages

NOTE: to code the stages check the official documentation
https://docs.aws.amazon.com/codepipeline/latest/userguide/action-reference.html

resource "aws_codepipeline" "pipeline" {
  name     = "pipeline"
  role_arn = "${data.aws_iam_role.pipeline_role.arn}"

  artifact_store {
    location = "${aws_s3_bucket.bucket-artifact.bucket}"
    type     = "S3"
  }
  # SOURCE
  stage {
    name = "Source"
    action {
      name             = "Source"
      category         = "Source"
      owner            = "AWS"
      provider         = "CodeCommit"
      version          = "1"
      output_artifacts = ["source_output"]

      configuration = {
        RepositoryName = "${var.repo_name}"
        BranchName     = "${var.branch_name}"
      }
    }
  }
  # BUILD
  stage {
    name = "Build"
    action {
      name             = "Build"
      category         = "Build"
      owner            = "AWS"
      provider         = "CodeBuild"
      version          = "1"
      input_artifacts  = ["source_output"]
      output_artifacts = ["build_output"]

      configuration = {
        ProjectName = "${var.build_project}"
      }
    }
  }
  # DEPLOY
  stage {
    name = "Deploy"
    action {
      name            = "Deploy"
      category        = "Deploy"
      owner           = "AWS"
      provider        = "ECS"
      version         = "1"
      input_artifacts = ["build_output"]

      configuration = {
        ClusterName = "clusterDev"
        ServiceName = "golang-Service"
        FileName    = "imagedefinitions.json"
      }
    }
  }
}
Enter fullscreen mode Exit fullscreen mode

DATA

This section is for using created IAM roles

data "aws_iam_role" "pipeline_role" {
  name = "codepipeline-role"
}

data "aws_iam_role" "ecs-task" {
  name = "ecsTaskExecutionRole"
}
Enter fullscreen mode Exit fullscreen mode

OUTPUTS

To get the ALB DNS and the CodeCommit Repository URL

output "repo_url" {
  value = aws_codecommit_repository.repo.clone_url_http
}

output "alb_dns" {
  value = aws_lb.app-lb.dns_name
}
Enter fullscreen mode Exit fullscreen mode

EXTRA VARIABLES

variable "repo_name" {
  type    = string
  default = "dev-repo"
}

variable "branch_name" {
  type    = string
  default = "master"
}

variable "build_project" {
  type    = string
  default = "dev-build-repo"
}

variable "uri_repo" {
  type = string
  #The URI_REPO value is in a TF_VAR in my PC
}
Enter fullscreen mode Exit fullscreen mode

STEP 03: HTTP Simple Server with GOLANG

This code is useful to get the PRIVATE IP of the ecs tasks

package main

import (
    "fmt"
    "log"
    "net"
    "net/http"
)

func main() {
    log.Print("HTTPserver: Enter main()")
    http.HandleFunc("/", handler)
    log.Fatal(http.ListenAndServe("0.0.0.0:5000", nil))
}

// printing request headers/params
func handler(w http.ResponseWriter, r *http.Request) {

    log.Print("request from address: %q\n", r.RemoteAddr)
    fmt.Fprintf(w, "%s %s %s\n", r.Method, r.URL, r.Proto)

    fmt.Fprintf(w, "Host = %q\n", r.Host)
    fmt.Fprintf(w, "RemoteAddr = %q\n", r.RemoteAddr)
    if err := r.ParseForm(); err != nil {
        log.Print(err)
    }
    for k, v := range r.Form {
        fmt.Fprintf(w, "Form[%q] = %q\n", k, v)
    }
    fmt.Fprintf(w, "\n===> local IP: %q\n\n", GetOutboundIP())
}

func GetOutboundIP() net.IP {
    conn, err := net.Dial("udp", "8.8.8.8:80")
    if err != nil {
        log.Fatal(err)
    }
    defer conn.Close()

    localAddr := conn.LocalAddr().(*net.UDPAddr)

    return localAddr.IP
}

Enter fullscreen mode Exit fullscreen mode

STEP 04: Dockerfile

  • This Dockerfile create a image with a HTTP Server with GOLANG
  • This file must be in the CodeCommit repository
FROM golang:alpine AS builder

ENV GO111MODULE=on \
    CGO_ENABLED=0 \
    GOOS=linux \
    GOARCH=amd64
WORKDIR /build
COPY ./HTTPserver.go .

# Build the application
RUN go build -o HTTPserver ./HTTPserver.go

WORKDIR /dist
RUN cp /build/HTTPserver .

# Build a small image
FROM scratch
COPY --from=builder /dist/HTTPserver /
EXPOSE 5000
ENTRYPOINT ["/HTTPserver"]
Enter fullscreen mode Exit fullscreen mode

STEP 05: Create TF_VAR

uri_repo

STEP 06: Create the Infrastructure

Commands

  • terraform init
  • terraform validate
  • terraform plan
  • terraform apply -auto-approve

When the creation is finished we get the OUTPUTS
outputs

STEP 07: Upload Dockerfile, Code and buildspect files to the CodeCommit repository

  • Clone the repository
    Alt Text

  • Copy buildspect.yml, Dockerfile and Golang Code to the cloned repository folder and then do a commit
    git

  • Git push to the CodeCommit Repository
    git push

STEP 08: Check the Pipeline

pipeline

  • When the "Build" stage is done, check the docker image in the ECR repository build

repo

STEP 09: Check the ECS Service

When the "Deploy" stage is done, check the Tasks in the ECS Service

deploy

service

tasks

STEP 10: Check the Target Group

target_group

STEP 11: Check the operation of the Application Load Balancer

alb1

alb2

FINAL STEP: Delete the Infrastructure

terraform destroy -auto-approve

Discussion (0)