DEV Community

Cover image for Building an Enterprise CI/CD Pipeline with Jenkins, Docker, Trivy, and GKE
Akshat Gautam
Akshat Gautam

Posted on

Building an Enterprise CI/CD Pipeline with Jenkins, Docker, Trivy, and GKE

Jenkins, the most popular open-source tool for build automation, plays a vital role in efficiently building, testing, and rolling out new applications across various environments.

One of those environments is Kubernetes, which is widely available as a managed service from all the leading cloud providers. In this article, we will explore how to build an automated, enterprise-ready CI/CD pipeline using Jenkins, Docker, and GitHub, and deploy our application to a Kubernetes Cluster on Google Kubernetes Engine (GKE).

Prerequisites

  • We need a running Jenkins server with SSH access to it. For this project, I have used Compute Engine to spin up an Ubuntu instance for me.
  • We need an active GCP Project and required access to it.
  • We have a Kubernetes cluster running on Google Kubernetes Engine (GKE). If you are new to GCP, You can use free-tier credits offered by the GCP for this demo.
  • We have a GitHub account for our code repository.

Pipeline Architecture Overview

Here’s a high-level look at the CI/CD pipeline that we'll be building:

  • Source Code Management: GitHub will serve as the version control system, where changes in the codebase trigger the pipeline execution.
  • Continuous Integration: Jenkins will be used to automate the entire build and deployment workflow. We'll also incorporate Trivy for security scans.
  • Continuous Deployment: The application will be built as a Docker image, pushed to Google Artifact Registry, and then deployed to GKE.
  • Monitoring and Notifications: Prometheus and Grafana will be set up to continuously monitor the health and performance of the deployed application. Email notifications will be configured to alert the team about build statuses and vulnerability scans.

Let's Dive in !


Setting Up Jenkins Server

Fire up an Ubuntu instance with a recommended disk space of 15GB and above.

Once the instance is up and running, SSH into it.

Image description

Run these commands on the machine to install Jenkins, Docker, Trivy and Kubectl



wget https://raw.githubusercontent.com/akshat2503/Jenkins-CI-CD/main/jenkins-server-setup.sh
chmod +x jenkins-server-setup.sh
./jenkins-server-setup.sh


Enter fullscreen mode Exit fullscreen mode

I am doing this tutorial in an Ubuntu Instance. For other Linux distributions, installation steps may differ. Please follow the official documentation Jenkins Installation

After installation completes, The output will be something like this:

Image description

Access the Jenkins server on the given link & authenticate using the first time password.

Install all suggested plugins

Image description

and finally create the admin user.

Prepare Jenkins

  • Install Required Plugins from Dashboard > Manage Jenkins > Plugins

Image description

  • Create credentials in Dashboard > Manage Jenkins > Credentials

1) Service Account from GCP

Create a service account, assign necessary permissions (like Artifact Registry Administrator, Kubernetes Engine Admin, Service Account User, etc.)

Create a key for it, and it will be downloaded on your local machine.

Image description

Save the credential in Jenkins. Choose kind as "Secret File" & upload the downloaded key.

Image description

2) Google App Password

Go to Manage Google Account > Security > 2-Step verification > App passwords

Create a new app password and copy it somewhere

Image description

Set username as the email address of the account from which the app password was generated & the password as the copied app password.

Image description


Explaining Pipeline Steps

1) Authenticate with Google Cloud

This step will use gcloud SDK to activate the service account & authorize Docker using the key you provided in Jenkins credentials.



withCredentials([file(credentialsId: 'service-acc-cred', variable: 'GC_KEY')]) {
sh '''
gcloud auth activate-service-account --key-file="$GC_KEY"
gcloud config set project $PROJECT_ID
gcloud auth configure-docker gcr.io -q
'''

}

Enter fullscreen mode Exit fullscreen mode




2) Clean Workspace

Cleans the workspace by deleting all the files from previous builds.



deleteDir()

Enter fullscreen mode Exit fullscreen mode




3) Checkout

Clones the GitHub repository to the Jenkins server for further processing.



sh 'git clone https://github.com/akshat2503/akshatgautam'

Enter fullscreen mode Exit fullscreen mode




4) File System Scan

Scans the downloaded GitHub repository for any possible vulnerability using Trivy and generates a report for it.



sh "trivy fs --format table -o trivy-fs-report.html ."

Enter fullscreen mode Exit fullscreen mode




5) Build Docker Image

Uses the Dockerfile present in the repository to build a Docker image.

You can also tag the image with some other parameter such as BUILD_NUMBER, I used latest just for simplicity.



sh 'cd akshatgautam && docker build -t gcr.io/$PROJECT_ID/portfolio-website:latest .'

Enter fullscreen mode Exit fullscreen mode




6) Docker Image Scan

Scans the generated Docker image for any possible vulnerability using Trivy and generates a report for it.



sh "trivy image --format table -o trivy-report.html gcr.io/$PROJECT_ID/portfolio-website:latest"

Enter fullscreen mode Exit fullscreen mode




7) Push Docker Image

The Docker image is then pushed to Google Artifact Repository.



sh 'docker push gcr.io/$PROJECT_ID/portfolio-website:latest'

Enter fullscreen mode Exit fullscreen mode




8) Deploy to GKE

This step fetches credentials for connection to your GKE cluster and then applies a deployment.yaml file.

The YAML file creates a deployment and a service of LoadBalancer type that will serve the application.



withCredentials([file(credentialsId: 'service-acc-cred', variable: 'GOOGLE_APPLICATION_CREDENTIALS')]) {
sh """
export PATH=
$PATH:$HOME/bin
gcloud config set project "
$PROJECT_ID"
gcloud container clusters get-credentials "
$CLUSTER_NAME" --zone "$CLUSTER_ZONE"
kubectl apply -f akshatgautam/k8s/deployment.yaml
"""

}

Enter fullscreen mode Exit fullscreen mode




9) Verify Deployment

Verifies the deployment using kubectl and logs the output to console.



sh 'kubectl get services'

Enter fullscreen mode Exit fullscreen mode




9) Email the final status

The post script sends an email with the status of the pipeline execution and also attaches both of the Trivy reports with the email.

Email may look like this


Pipeline Creation

Let's get started with setting up your Jenkins pipeline:

  • Go to Dashboard > New Item

Name: jenkins-gcp-integration-pipeline
Project Type: Pipeline

  • Scroll down and add the pipeline script, Get the script from here

Image description

Our pipeline is set up, but not yet ready to build. Mail configuration to receive mail updates is still pending !

  • Go to Dashboard > Manage Jenkins > System, scroll down to find Extended E-mail Notification

SMTP Server: smtp.gmail.com
SMTP Port: 465
Credentials: Choose the one we previously created
Check "Use SSL"

Image description

  • Now just below this, You'll find E-mail Notification

SMTP Server: smtp.gmail.com
Check "Use SMTP Authentication"
Username: Your email
Password: App Password that we copied earlier
Check "Use SSL"
SMTP Port: 465

Image description

Click "Save"

Voila! Your Jenkins pipeline is now ready


Pipeline Testing

  • Go to Dashboard > jenkins-gcp-integration-pipeline and click Build Now

A build will trigger within a second. All the pipeline steps will be followed sequentially.

Here are the results:

Image description

Image description


Verifying Deployment

  • Go to GKE > Workloads

Image description

  • Go to GKE > Gateways, Services & Ingress. Your application should be listed there.

Image description


Conclusion

At this point, we have successfully created a simple CI/CD pipeline between Jenkins, GitHub, Artifact Registry and one Kubernetes cluster running on Google Kubernetes Engine.

You can further enhance this pipeline by adding some features like:

  • GitSCM Polling/Webhook to trigger build on code commit
  • Including integration, performance tests on the code
  • Multi-cluster deployment
  • Error handling and retry mechanisms
  • and what not ...

Monitoring through Prometheus & Grafana will be documented in another article to keep this one concise.

Drop your questions in comments and I will be very happy to help !

Happy Learning !

Top comments (6)

Collapse
 
aman-giri profile image
Aman Giri

The article is top-notch!
I have a simple query: Is it feasible to use GitHub Actions instead of Jenkins? Do GitHub Actions have the same capabilities?

Collapse
 
akshat_gautam profile image
Akshat Gautam

Thanks for your appreciation !
Glad you liked it.

Yes, Of course. We can surely use GitHub Actions for this. GitHub Actions also has he same capabilities, It also has a very tight integration with GitHub.
Jenkins can be preferred for more complex workflows

I may publish an article on implementing such a pipeline on Actions too.
Follow & Stay Tuned

Collapse
 
shoaib1729 profile image
Shoaib Akhtar

Badhiya.

Collapse
 
akshat_gautam profile image
Akshat Gautam

Shukriya :)

Collapse
 
shubhanshuk profile image
Shubhanshu Kakkar

Amazing πŸ‘ bro

Collapse
 
akshat_gautam profile image
Akshat Gautam

Thank You very much :)