In this post we are going to cover setting up CI/CD pipeline using google cloud build. If you need to set up a GCP account or need to deploy your application, I already wrote something that should get you up and running https://dev.to/ryanmercadante/deploy-your-mern-stack-application-to-google-app-engine-2g2c.
Table of Contents
- What is CI/CD?
- Setup Google Cloud Build
- Creating Build Triggers
- Setting up Cloud KMS
- Encrypting our Environment Variables
What is CI/CD?
CI/CD stands for continuous integration and continuous deployment/delivery depending on who you ask. It is the process of creating pipelines to automate certain tasks such as building, testing, merging, and deploying your code. Red Hat has a great article going in depth on the nuances between the integration, delivery, and deployment phases https://www.redhat.com/en/topics/devops/what-is-ci-cd, and I would highly recommend reading it if you're interested in understanding it better.
Setup Google Cloud Build
First head over to the GCP console and go to APIs and Services. You'll want to search the API library for Cloud Build API and enable it.
Next, go to Cloud Build in the tools section and click on settings. My application is deployed to app engine, so I am going to enable app engine. You should enable Cloud KMS as well, but we'll cover that more later in the tutorial.
If your application is deployed to app engine as well, you will also need to enable the App Engine Admin API back in the API library.
With that set up, let's head back to the Cloud Build page and go to Triggers. The first thing we need to do is to connect our repository.
Select where you keep your source code (I use GitHub).
Next it will ask you to authenticate with whichever provider you chose and then to select repositories to connect to cloud build. You should see a link to select repositories on GitHub (or whichever you use). You can select specific repos or all of them. I chose to only select the two projects that will be using cloud build at this moment. One is a react application and the other is an express API.
Once you click save you'll be redirected back to the page you were on. Select the repos you just gave permission to, and the checkbox saying you understand the warning they provide.
The last step to create a push trigger is optional and we are going to handle that in the next section, so skip that for now.
Creating Build Triggers
The next step in the process is to create our build triggers. Cloud Build offers 120 minutes of free build time per day, but after that it is $0.003 per minute. This is a very generous free tier, and for personal learning projects, even if you did go over the free tier limit it is very unlikely you would rack up any excessive cost. With this in mind though, we are only going to create one trigger for each repo I added. The trigger will install dependencies, build our app, and deploy it. You can also include other build steps like linting, testing, etc.
With that out of the way let's build our first trigger. Go to the cloud build dashboard and click the button to set up build triggers.
We are going to name the trigger push-to-master, and this needs to be unique within the project. There are three trigger types; branch, tag, and pull request. In this tutorial we are going to use the branch trigger type. In the branch section just type in master so it knows to only trigger when you push to the master branch. We are going to ignore a few optional sections which allow you to trigger based on changes to specific files. For our build configuration we are going to use a cloud build configuration file; cloudbuild.yaml. It should default the file location to the root of your project. This is what our file looks like.
steps:
- name: "gcr.io/cloud-builders/npm"
args: ["install"]
- name: "gcr.io/cloud-builders/npm"
args: ["run", "build"]
- name: "gcr.io/cloud-builders/gcloud"
args: ["app", "deploy"]
timeout: "1600s"
This build uses two different cloud builders; npm and gcloud. Cloud builders are just container images that allow you to run commands within the context of the builder. The args field takes a list of arguments that the builder will run. So using the npm builder with the arguments run and build is the same as running npm run build in your terminal. If you click on the history tab, you should see a list of your previous builds.
You can further inspect the build process by clicking on the build Id in the build column.
Clicking into these tabs will give you more insight into what was going on behind the scenes. This is especially useful if your build failed.
That was the build trigger for my react app. The cloudbuild.yaml is going to look a little bit different for my express API because I am using environment variables that need to be encrypted. In order to do this properly, we are going to need to enable the Cloud Key Management Service (KMS) API.
Setting up Cloud KMS
Cloud KMS will allow us to create encryption keys to protect secrets and other sensitive data that we need to store in the Google Cloud Platform. More specifically, we are going to use it to encrypt our .env file because that has our database URI.
Search for Cloud Key Management Service (KMS) API in the API library and enable it. Before we can create a cryptographic key, we need to create a keyring. We are going to be using a gcloud command for this, otherwise you can create a keyring using the console.
Run gcloud kms keyrings create cloudbuild-keys --location global in your terminal to create your keyring. I named mine cloudbuild-keys but you can name it whatever you'd like. You should see this in the console.
Next we are going to create our key. Run the command gcloud kms keys create api-key --location global --keyring cloudbuild-keys --purpose encryption. You should see the key in the keyring you created.
If you want to use the console to create your keyring and keys, it's pretty straightforward and I have provided pictures to show you the console equivalent of our gcloud commands.
Encrypting our Environment Variables
Next, we are going to encrypt our .env file so that we can use it in our build. Just to reiterate, I have to do this for my deployed API because my environment variables include my database URI, which should be kept private. I didn't have any sensitive information for the client which is why none of these steps were necessary.
Run the following command in the root folder of your project but make sure to change out my project specific information with your project specific information.
gcloud kms encrypt \
--plaintext-file=config/config.env \
--ciphertext-file=config/config.env.enc \
--location=global \
--keyring=cloudbuild-keys \
--key=api-key
It is safe to commit your encrypted file to source code, but make sure you add your unencrypted file to your .gitignore.
Now that we have successfully encrypted our sensitive information, we can use it in our CI/CD pipeline. My cloudbuild.yaml file for my API looks like this. Once again you'll want to remove my project specific information and add yours instead. This will decrypt our config.env.enc at build time so that we can use the decrypted file in our workspace.
steps:
# Decrypt our env file
- name: gcr.io/cloud-builders/gcloud
args:
- kms
- decrypt
- --ciphertext-file=config/config.env.enc
- --plaintext-file=config/config.env
- --location=global
- --keyring=cloudbuild-keys
- --key=api-key
# Deploy our app
- name: "gcr.io/cloud-builders/gcloud"
args: ["app", "deploy"]
timeout: "1600s"
Hopefully all of these steps worked for you and you have a fully functioning CI/CD pipeline for your application! If anything was unclear or something isn't working right, leave me a comment down below or message me and I'll do my best to help you out. If you would like to get in touch for any reason, feel free to connect with me on LinkedIn, follow me on Twitter, or send me an email. Thanks for reading!
Top comments (0)