AWS CICD PIPELINE

Setting Up a GitHub Repository for Your Python Application

If you don't already have a GitHub repository, follow these steps to create one:

• Sign In to GitHub:

  • Go to github.com and log into your account.

• Create a New Repository:

  • Click the "+" button in the top-right corner.
  • Select "New repository."

• Configure the Repository:

  • Name: Enter a name for your repository.
  • Description: (Optional) Add a description for your repository.
  • Visibility: Choose between public or private based on your preference.
  • Initialize: Select "Initialize this repository with a README."

• Create the Repository:

  • Click the "Create repository" button.

• Download all the files from https://github.com/SUBHAM-NANDI/Project2-AWS-END-TO-END-CICD on local machine and upload it to the new repository.

Configuring AWS CodeBuild for Your Python Application

In this step, we will configure AWS CodeBuild to build and package our Python application for deployment. Follow these steps:

• Navigate to AWS CodeBuild:

  • Go to the AWS Management Console.
  • Search for and select "CodeBuild."

• Create a New Build Project:

  • Click on the "Create build project" button.
  • Enter a name for your build project.

• Configure the Source Provider:

  • For the source provider, choose "GitHub."
  • Connect the GitHub account with the AWS CodeBuild.

• Configure the Build Environment:

  • Operating system - Ubuntu with the latest image
  • Choose the runtime (Python 3.12).

• Create a new role:

  • Select a New Service Role
  • Give a name to the Service Role
  • You can also manually create an IAM role and select it here.

• Specify Build Commands:

  • Select Insert build commands
  • Click on Switch to editor

version: 0.2

#env:
#  parameter-store:

phases:
  install:
    runtime-versions:
      python: 3.12
  pre_build:
    commands:
      - echo "Installing dependencies..."
      - pip install -r simple-python-app/requirements.txt
  build:
    commands:
      - echo "Running tests..."
      - cd simple-python-app/
      - echo "Building Docker image..."
      - echo "<>"
      - docker build -t "<>"
      - docker push "<>"
  post_build:
    commands:
      - echo "Build completed successfully!"

NOTE - We will set the env variables in the parameter store (Systems Manager), build and push cmds later.

Set Env variables in parameter store:

• Navigate to the Parameter Store:

  • Go to the AWS Management Console.
  • Search for and select "Systems Manager."
  • In the left-hand navigation pane, under "Application Management," select "Parameter Store."

• Create a New Parameter:

  • Click on the "Create parameter" button.
  • Create 3 parameters namely /myapp/docker-credentials/username, /myapp/docker-credentials/password and /myapp/docker-registry/url.

• Define the Parameter Details:

  • Name:
    • Enter a name for your parameter.
  • Description:
    • (Optional) Enter a description for your parameter.
  • Tier:
    • Choose the parameter tier (Standard or Advanced).
  • Type:
    • Choose "String" for plain text or "SecureString" if you want to encrypt the value.
  • Value:
    • Enter the value for your parameter (e.g., your Docker username or password).

• Specify Advanced Settings:

  • KMS Key Source:
    • If you chose "SecureString" as the type, specify the KMS key to encrypt the value. You can use the default key or choose a custom key.
  • Data Type:
    • Leave it as "text" unless you have specific requirements to use other data types.
  • Tags:
    • (Optional) Add tags to help organize and manage your parameters.

• Review and Create:

  • Review the details you have entered.
  • Click on the "Create parameter" button.

Accessing Parameters in CodeBuild

To use these parameters in your CodeBuild project:

• Add Parameter Store Permissions:

  • Ensure that your CodeBuild service role has permission to access the parameters. Attach the AmazonSSMReadOnlyAccess policy or a custom policy with necessary permissions to the role.

• Final changes in buildspec.yml:
  
  

version: 0.2

env:
  parameter-store:
    DOCKER_REGISTRY_USERNAME: /myapp/docker-credentials/username #this consists the username of Docker Hub
    DOCKER_REGISTRY_PASSWORD: /myapp/docker-credentials/password #this consists the password of Docker Hub
    DOCKER_REGISTRY_URL: /myapp/docker-registry/url #this consists the location/url(docker.io) of Docker Hub
phases:
  install:
    runtime-versions:
      python: 3.12
  pre_build:
    commands:
      - echo "Installing dependencies..."
      - pip install -r simple-python-app/requirements.txt
  build:
    commands:
      - echo "Running tests..."
      - cd simple-python-app/
      - echo "Building Docker image..."
      - echo "$DOCKER_REGISTRY_PASSWORD" | docker login -u "$DOCKER_REGISTRY_USERNAME" --password-stdin "$DOCKER_REGISTRY_URL"
      - docker build -t "$DOCKER_REGISTRY_URL/$DOCKER_REGISTRY_USERNAME/simple-python-flask-app:latest" .
      - docker push "$DOCKER_REGISTRY_URL/$DOCKER_REGISTRY_USERNAME/simple-python-flask-app:latest"
  post_build:
    commands:
      - echo "Build completed successfully!"

• Review and Create the Build Project:
  • Review all the build project settings.
  • Click on the "Create build project" button.

CodeBuild may fail in the following circumstances:

1. echo "$DOCKER_REGISTRY_PASSWORD" | docker login -u "$DOCKER_REGISTRY_USERNAME" --password-stdin "$DOCKER_REGISTRY_URL"
2. give AmazonSSMReadOnlyAccess to IAM role policy.
3. A dot(.) after $DOCKER_REGISTRY_URL/$DOCKER_REGISTRY_USERNAME/simple-python-flask-app:latest" cmd may also lead to failure.
4. In Codebuild, grant access to the Privileged section.

Create an AWS CodePipeline

• Navigate to AWS CodePipeline:

  • Log in to the AWS Management Console.
  • In the Services menu, search for and select CodePipeline.

• Create a Pipeline:

  • Click on the Create pipeline button.

• Pipeline Settings:

  • Pipeline name: Enter a unique name for your pipeline (e.g., PythonAppPipeline).
  • Service role: Choose "New service role" to allow CodePipeline to create a service role, or select an existing one.
  • Artifact store: Use the default S3 bucket or select a custom location if needed.
  • Click Next.

Create an EC2 instance

Launch an Instance
Launch Instance: In the EC2 Dashboard, click on the Launch Instance button.

Select an AMI: Choose an Amazon Machine Image (AMI) that suits your needs (Ubuntu Server AMI).

Instance Type: Select an instance type. The t2.micro instance type is suitable and is eligible for the AWS Free Tier.

Tags: Add a tag to identify your instance.

Key Pair: You will be prompted to select an existing key pair or create a new one. This key pair is used to SSH into your instance.

• Existing Key Pair: Select an existing key pair from the dropdown.
• New Key Pair: Create a new key pair, download the private key file (.pem), and keep it secure.

Configure Instance Details: Select the other configuration details as required and launch the instance.

Access Your Instance

1. View Instances: Click on the View Instances button to see your running instances.
2. Instance State: Wait until the instance state changes to running.
3. Public IP: Note the public IP address of your instance.

Connect to Your Instance

• SSH Connection:
  • Open a terminal on your local machine.
  • Use the following command to connect, replacing path/to/your-key.pem with the path to your key file and ec2-user with the appropriate user (e.g., ubuntu for Ubuntu instances):

 ssh -i path/to/your-key.pem ec2-user@your-instance-public-ip

• Verify Connection: If the connection is successful, you will see a welcome message and a prompt from your EC2 instance.

IMPORTANT - We have to give code-deploy permissions to the EC2 instance.

Install the agent

To install the agent, follow the steps from the doc link given below as it is.
https://docs.aws.amazon.com/codedeploy/latest/userguide/codedeploy-agent-operations-install-ubuntu.html

Restart the agent if any configuration changes are made using the commands below.

systemctl status codedeploy-agent
sudo service codedeploy-agent restart

Install docker on the EC2 instance

The docker container will not run unless we install docker on the EC2 instance.
sudo apt install docker.io -y

Create appspec.yml file

Create an appspec.yml file with the below configurations in root project folder.

version: 0.0
os: linux

hooks:
  ApplicationStop:
    - location: scripts/stop_container.sh
      timeout: 300
      runas: root
  AfterInstall:
    - location: scripts/start_container.sh
      timeout: 300
      runas: root

Create scripts

Write two scripts, namely start_container and stop_container in the scripts folder in root project folder.

start_container

!/bin/bash
set -e

# Pull the Docker image from Docker Hub
docker pull subhamnandi/simple-python-flask-app:latest

# Run the Docker image as a container
docker run -p -d 5000:5000 subhamnandi/simple-python-flask-app:latest

stop_container

!/bin/bash
set -e

# Stop the running container (if any)
containerid= `docker ps | awk -F " " '{print $1}'`
docker rm -f $containerid

Go to CodePipeline and release the changes.