1. Definitions

ECS

• ECS stands for Elastic Container Service
• ECS allows you to launch Docker containers(Tasks) on AWS

Launch Types

It is very important to understand how ECS works depends on its launch type.

ECS needs physical machines that can run Docker daemon. There are two ways to launch ECS

1. EC2 Launch Type

• EC2 launch type runs Docker containers on EC2 instances. And each EC2 instances can run several Tasks.
• Depends on the Auto Scaling Group set up, more EC2 instances will be automatically launched to run more Tasks.
• The launched EC2 instances MUST have ECS Agent to run Tasks. AWS will automatically launch AMI that ECS Agent is already installed.

2. Fargate Launch Type

• There is no EC2 instances to manage. AWS will manage.
• To run more tasks, simply need to add more tasks. There is no need to know which exact machine will be running the tasks.

So which one we should use? It really depends on the situation, but normally EC2 launch type is more cost efficient compared to Fargate launch type. Typically, a service will end up charging more when the service is more easy to handle and user friendly.

Check out this blog post if you want to know more about comparison between those two services. Link

Tasks

• A single Task simply means a single Docker container.
• More precisely, a single Docker container will be running on top of a Task. And the Task will only contain one Docker container.
• Each Task can have different IAM Role.

Clusters

• A Cluster can manage several EC2 machines or Fargate Tasks.
• Notice that each EC2 machine can run several Tasks.

2. Usage

• Process batch jobs or data to convert and store them to database (e.g.DynamoDB).
• Want to deploy multiple containers for better performance for your applications.

ECS vs Lambda?

• When it comes to processing batch jobs, ECS and Lambda does pretty much the same thing. Here is a good blog post explains the difference between two and when to use them Link

• To summarize the blog post

  • ECS provides environment flexibility & task scheduling.
  • Lambda is good for processing batch jobs that take less than 15 minutes. Also, can be more time & cost efficiency depends on what you want to do

3. Experiment Overview

Image processing with ECS

User interaction

1. Upload image files to S3. This will be done manually.
2. Launch the prepared ECS Task using Fargate.

Inside of the Fargate Task Link for the GitHub Repo

1. The container will run Python scripts as it is configured.
2. Download all the images in /new folder in the S3 bucket with the specified region and name.
3. Process the images. For this tutorial, I did not do anything for the image processing job.
4. Move all image files in /new to /old

3. Conclusion

The rest of implementation will be continued in [AWS Experiment] 8 - AWS ECS Deep Dive (Part.2)