Configuring An End-to-End CI/CD Pipeline Using CircleCI, Ansible and Cloudformation

I recently completed a project in the udacity cloud DevOps Nanodegree program , during which I implemented CI/CD for the UdaPeople product, a revolutionary concept in Human Resources which promises to help small businesses care better for their most valuable resource: their people. During the course of the project, I demostrated mastery of the following objectives.

1. Utilizing Deployment Strategies such as CircleCI and a version control system e.g Github, to design and build CI/CD pipelines that support Continuous Delivery processes.
2. Utilizing a configuration management tool such as Ansible to accomplish deployment to cloud-based servers.
3. Surface critical server errors for diagnosis using centralized structured logging and monitoring such as prometheus.

The Beauty of CI/CD

In order to understand the beauty of CI/CD, let's define each term.
Continuous Integration is a development practice that requires developers to integrate code into a mainline as frequently as possible, at least once a day. An automated build that compiles the code then verifies each check-in and runs the set of automated tests against it so that teams can quickly detect problems.
Continuous Delivery is the process of getting changes of all types, including configuration changes, bug fixes, experiments and new features into production or into the hands of users in a sustainable way. When a team of developers implements continuous delivery, the mainline is in a deployable state and anyone can deploy it to the production anytime with the click of a button. When the button is clicked, an automated pipeline gets triggered. The significant element to achieve continuous delivery is automation.
Continuous Deployment is a step up from Continuous Delivery where every change in the source code is deployed to production automatically without requiring explicit approval from a developer. A developer’s role usually ends at checking a pull request from a teammate and merging it to the master branch. Continuous Integration/Continuous Delivery takes it from there by executing all automated tests and deploying the code to production while keeping the team updated about the outcome of every event.
Continuous Integration, Continuous Delivery and Continuous Deployment are like vectors with the same direction but different magnitude. All three terms aim to make the software development and release process more robust and quicker.

CircleCI, Ansible and Cloudformation

The udapeople product relied on automation tools such as circleci, ansible and cloudformation in order to reduced the workload on it's developers and guarantee its sustainability.
CircleCI is the continuous integration & delivery platform that helps the development teams to release code rapidly and automate the build, test, and deploy. After repositories on GitHub or Bitbucket are authorized and added as a project to circleci.com, every code commit triggers CircleCI to run jobs defined in it's .circleci/config.yml file.
Ansible is an open-source automation tool, or platform, used for IT tasks such as configuration management, application deployment, intraservice orchestration, and provisioning. Ansible can automate IT environments whether they are hosted on traditional bare metal servers, virtualization platforms, or in the cloud. It can also automate the configuration of a wide range of systems and devices such as databases, storage devices, networks, firewalls, and many others.
AWS CloudFormation provides a common language for you to describe and provision all the infrastructure resources in your cloud environment. It allows you to use a simple text file to model and provision, in an automated and secure manner, all the resources needed for your applications across all regions and accounts. Check out this post on deploying a high availability webapp using cloudformation.

Project steps

1.Setup - AWS

• Create and download a new key pair in AWS EC2. Name this key pair whatever name you wish ( I named mine "udapeople.pem").
• Add a PostgreSQL database in RDS that has public accessibility. This tutorial may help. As long as you marked "Public Accessibility" as "yes", you won't need to worry about VPC settings or security groups. Take note of the connection details, such as:

Endpoint (Hostname): database-1.ch4a9dhlinpw.us-east-1.rds.amazonaws.com
Instance identifier: database-1 //This is not the database name
Database name: postgres (default)
Username: postgres
Password: mypassword
Port: 5432

• Note that the AWS wizard will create a default database with name postgres. If you wish to give another name to the initial database, you can do so in the additional configuration as shown in the snapshot below.

Verify the connection to the new database from your local SQL client, using this tutorial.

2.Set up - CircleCI

• Set up project in CircleCI.

• Add the SSH key (*.pem) to CircleCI.

• Add the environment variables to CircleCI by navigating to {project name} > Settings > Environment Variables.

AWS_ACCESS_KEY_ID=(from IAM user with programmatic access)
AWS_SECRET_ACCESS_KEY= (from IAM user with programmatic access)
AWS_SESSION_TOKEN= (from IAM user with programmatic access)
AWS_DEFAULT_REGION=(your default region in aws)
TYPEORM_CONNECTION=postgres
TYPEORM_MIGRATIONS_DIR=./src/migrations
TYPEORM_ENTITIES=./src/modules/domain/**/*.entity.ts
TYPEORM_MIGRATIONS=./src/migrations/*.ts
TYPEORM_HOST={your postgres database hostname in RDS}
TYPEORM_PORT=5432 (or the port from RDS if it’s different)
TYPEORM_USERNAME={your postgres database username in RDS}
TYPEORM_PASSWORD={your postgres database password in RDS}
TYPEORM_DATABASE=postgres {or your postgres database name in RDS}
ENVIRONMENT=production

3.When a change has been pushed to GitHub non-main branch, it will test and scan the backend and frontend of the app.

4.When a change has been pushed to GitHub main branch, it will trigger the pipeline and deploy the backend and frontend of the app to aws.

5.The app allows you to add new employees. The frontend URL can be obtained through S3 and CloudFront. The backend URL can be seen through EC2.

Conclusion

Considering the udapeople HR product as a case study, the benefits of an automated CI/CD pipeline range from practical considerations like code quality and rapid bug fixes, to ensuring you’re building the right thing for your users and improving your entire software development process.

Despite the name 'DevOps' suggesting a focus on developer and operations teams, building a CI/CD pipeline provides an opportunity for collaboration across a whole range of functions. By streamlining the steps to release your product, you provide your team with more insights into how your product is used and free up individuals’ time so they can focus on innovation.

Did you enjoy reading the article? Drop a comment.