Wednesday Solutions

Posted on Feb 22, 2022 • Edited on Feb 23, 2022

How to use Copilot to deploy projects on ECS

#githubcopilot #aws #devops #javascript

AWS Copilot is a tool that allows you to create and manage your databases, ECS applications, and other resources. All from the command line.

Very often, the success of your product is dependent on the number of users your product is able to reach. AWS Services like ECS enables you to achieve high scale while keeping costs low. ECS dynamically spawns up new instances of your application to keep up with high-load demands. It also automatically scales down the number of instances when the demand reduces.

In order to make your application available to your users, ECS must integrate with and leverage a lot of other AWS infrastructure services like the Elastic Container Registry, Target Groups, VPC, etc. This process of creating and mapping the services must be repeated for each of the environments that your application is being deployed in. And if you’re truly looking to handle scale and manage costs, you’re looking at building micro-services which entails repeating these steps for each of the environments for every service!!! Phew! I’m getting tired just thinking about it.

AWS Copilot removes the overhead of creating and mapping each of these AWS services manually, and allows us to spend time solving real world business problems. With some bash sorcery we’re able to setup and deploy the following infrastructure components using a single command

Elastic Container Registry
ECS Cluster
ECS Service
ECS Task definition
Aurora serverless cluster
Security groups, VPCs, and the works.

This tutorial assumes a good understanding of

Docker
Bash

In this tutorial, I will walk you through how to publish and update a node application using 4 simple steps

Install and configure AWS CLI
Install Copilot CLI
Creating the infrastructure and deploying your application on AWS using copilot.
Deploy an update to your application

Step 1 - Install and configure AWS

Install the AWS CLI

To install the CLI follow the steps mentioned here: AWS CLI installation instructions

Configure the AWS CLI

To configure the CLI please follow these steps:

Create a User Group named Administrators. Set the AdministratorAccess permission policy.
Navigate to the Users tab and add a new user with username Administrator.
Select the AWS Credential Type as Access Key - Programmatic access.
Add the user to the Administrators group.
Save the access token ID and the secret access key. Here is a video showing these steps.

Open up the terminal and enter the following command.

aws configure

‍

Step 2 - Install Copilot CLI

Copilot CLI Installation instructions

Copilot CLI will use the credentials configured for AWS CLI.

Step 3 - Install & Start Docker

If you haven’t already, install docker. Please follow the steps mentioned here. Next is to make sure docker is installed and is running. Copilot will use docker to create images of your application.

Step 4 - Starter Project

Open the terminal and clone the starter project using this command.

git clone https://github.com/wednesday-solutions/aws-copilot-node-demo-starter.git

This repository contains a containerized application. Peek around the repo to know the files and folders.

Step 5 - Setup ECS

Open the terminal, navigate to the cloned repository and run setup-ecs.sh with the app and the environment name.

Once you run the script you’ll be asked if you’d like to deploy to a test environment, say no. We’ll get to that in a bit.

Copilot will use the default profile for credentials and configuration defined in ~/.aws/credentials and ~/.aws/config

Let’s go through the setup-esc.sh file in detail.

echo "name of the service is: $1 $2"

# 1 
copilot init -a "$1" -t "Load Balanced Web Service" -n "$1-$2-svc" -d ./Dockerfile

# 2
copilot env init --name $2 --profile default --default-config

# 3
copilot storage init -n "$1-$2-cluster" -t Aurora -w 
"$1-$2-svc" --engine PostgreSQL --initial-db "$1_$2_db"

# 4
copilot deploy --name "$1-$2-svc" -e "$2"

‍

This line creates the manifest.yml that contains the instructions to setup a Load Balanced Web Service. It points to the ./Dockerfile that we have in the repo

2.
This line creates the Elastic Container Repository, KMS Key, necessary IAM Roles, public & private subnets, a VPC, etc.

3.
Creates a cloudformation template to provision an Aurora serverless cluster with a PostgreSQL engine.

Builds and deploys the docker container.

And just like that your application has been deployed! On successful execution of the the script it will give out the public DNS of your application.

Use the public DNS logged at the end to check out your deployed application. You can obtain the public URL from the associated load balancer.

Step 6 - Copilot

Open the project in your favorite editor. You’ll notice a new folder called copilot is created. The setup-ecs script also runs a command to initialize copilot.

This folder contains the ECS Services manifests and their add-ons. Here is a high level description of each file.

manifest.yml: This file consists of configurations for: Docker Task definition Docs for the manifest can be found here
Addons: Add-ons are CloudFormation templates. To add more resources for your application, make changes in the template. CloudFormation template reference can be found here.
Database configuration: This folder contains the RDS database configuration. For the database the following resources were created.

Outputs

The database template injects the environment variables defined in Outputs.

Database connection details will be available in PLAYGROUNDDEVELOPCLUSTER_SECRET

Connecting to the database

Let us deploy an update after connecting to the database using the above environment variable.

....
  try {
        // using the database credentials from the injected output variable
    const { username, host, dbname, password, port } = JSON.parse(
                process.env.PLAYGROUNDDEVELOPCLUSTER_SECRET
        );
    console.log({ username, host, dbname, password, port });
    pg({
      database: dbname,
      user: username,
      password,
      port,
      host
    })
      .then(console.log)
      .catch(console.error);
  } catch (err) {
    console.error('err', JSON.stringify(err));
  }
....

Deploying an update

After making changes, run the update-ecs script with the app name and the environment. An updated image of the container will be built and pushed to ECR. Once the new revision of the task definition is stable, the previous version will be brought down.

# deploy an update after the changes to the service
copilot deploy --name "$1-$2-svc" -e "$2"

Post Install

Once the service is up and running, the following might be handy.

Connect to the database

Security groups attached to the database resource will not allow for connections outside the service. Follow this guide to use an ec2 instance to connect to your database.

Obtain the database credentials

Obtain the cluster secret created by Copilot from the Secrets manager console.

Troubleshooting

In case of deployment failures, take a look at CloudFormation event logs. Copilot uses CloudFormation to manage resources.

Where to go from here

Congratulations! You have now automated the creation and updation of all of the ECS infrastructure. Take a look at the final output here.

This tedious and cumbersome process can now be done friction-free by running a single command. Use it to build your next side project, or amaze your coworkers with these newly acquired DevOps chops!

I hope you enjoyed this tutorial as much as I enjoyed writing it. If you think this is helpful please share and leave comments. Alternatively you can tweet your comments at us here.

Author:
Christin Itty

DEV Community