Automated CI/CD Pipeline for Web Applications Using Jenkins and GitHub

This is my portfolio project for the foundations year at ALX Africa and Holberton School.
It is an Automated CI/CD Pipeline built with Jenkins integrating with GitHub using web-hooks and integrating with Docker.
The main purpose of the project is to build an automated seamless pipeline for web applications using Continuous Integration or Continuous Delivery for web applications. Code changes and updates can be made for scalable applications using the pipeline without in any sort running the application down and also be tested and deployed using Docker.

Team Members, Roles and Timeline:

• Members: This is a sole project built by Emmanuel Madubuike.

• Roles:

  • Project Manager: This project was supervised by me.
  • Developer: This project was developed also by me.
  • Tester: All tests(Integration) was done by me.

• Timeline: This project was built on a 3 weeks time-frame.

This project was built for Software Engineers and DevOps Engineers to provide Continuous Integration or Continuous Delivery of scalable applications to users. Personal focus for this project was configuring the Jenkins pipeline, integrating with GitHub with web-hooks and also deploying on a Docker server.
I have always been a candid lover of DevOps so i thought it will be a great idea to work on a project relating to the field. I had earlier decided that I will work on a CI/CD Pipeline for my portfolio project but I had no prior idea as at then. So I made a through research on projects I can work on with a short time-frame on the related field and I came up with this. Throughout the timeline for this project, I came across some technologies I haven't used or heard before and it helped me deepen my understanding on them. I would have never guessed that 9 months later since my first day at ALX Africa that I will work on a sort of a project as this.

Architecture

Attached above is the architecture of the project. Code updates for the application will be pushed to GitHub which will automatically trigger a build process on Jenkins using web-hooks. Upon successful build process, the Jenkins server is configured to test and deploy to Docker either using the web interface or writing a Jenkinsfile for build, test and deploy operations.

Technologies

The technologies used for this project are as follows:

• Jenkins: This is used to create the CI/CD Pipeline for the project.

• GitHub: This is the version control for the source code. Written code and code changes are pushed to it.

• Docker: Deployment of the application in containers for test and deployment purposes.

• Flask: This was used to write a sample web application with a few endpoints to test the pipeline.

• AWS web servers (EC2): Jenkins server and Docker server were installed on it to provide public internet access.

Key Features

Some of the key features in this project includes:

• Webhooks: Webhooks serves as a trigger between Jenkins and GitHub, keeps track of code changes in the version control and aids in seamless Continuous Delivery.

• Docker Integration: Direct integration of Docker from Jenkins to test and deploy applications through containerization. On the image below, it shows the configuration method in which Docker can be set for use.

• Jenkins Configuration: Jenkins can be configured for certain tasks and certain builds depending on the job given. It is a nice feature as it enables to build, test and deploy on the pipeline.

Technical Challenges

On course of the project, I faced two major technical challenges of which I will list below:

• Webhooks: This was the first major challenge I faced. At the beginning of the project, I had initially installed Jenkins server on my local machine which means the Jenkins server will be running on localhost. As it worked on localhost, I tried to configure webhooks for Jenkins but it wasn't working. I made a research and discovered Ngrok, which allows localhost to be seen on the public internet but Ngrok came with a difficulty, the timeout session. Figuring out this won't work efficiently, i made more extensive research and found the solution by making use of web servers for installation of Jenkins and Docker. After configuration and removal of firewall for the port, it started to work out well.

• Docker Integration: This was also a major challenge i faced. I had initially installed Docker on my local machine so I could integrate it with my Jenkins server. On course of integrating, each deployment process I tried to run kept failing. I had to install it on my second web server and made it possible so I can ssh into the web server from my first web server. With the above method, it began to work perfectly.

Conclusion

As an engineer, so far on this project, I became more familiar with the above-mentioned technologies, of which I had no idea about. From integration to configuration of the pipeline. This project deepened my understanding of DevOps, CI/CD, Docker containers and Pipelines.

About the Author

Emmanuel Madubuike is a Software Engineer specializing with BackEnd and DevOps Engineering. Currently a student of ALX Africa and Holberton School. You can connect with me via the following handles:

• LinkedIn: https://www.linkedin.com/in/madubuike-emmanuel

• GitHub Project Repository: https://www.github.com/NuelUzoma/Portfolio-Project

• Project Landing Page: https://emcool17.wixsite.com/portfolio

• Jenkins server web interface: http://107.23.96.11:8080/job/Portfolio-Project