Side note: I'm an intermediate Sys admin and have been building my skills to be proficient. If there's a mistake, please let me know.
Warning: Using Terraform v0.13
What is cloud run? Cloud run is a fully self-service managed infrastructure that is capable of automated scaling.
cloudrun
What is TerraForm? Terraform is a multi-cloud infrastructure as code that is open-source. This eliminates human error and reduces time.
TerraForm
In this how-to article, I'll show you how to deploy your own API on Cloud Endpoints and use the cloud run service for both Application and API gateway ESPv2 image with Terraform and cloud build.
I followed this tutorial and replicated it through CI/CD pipeline GettingStarted CloudEndpoints
Prerequisites
- Create Google Cloud Project and account GCPProject
- Create Github Account and new repository on Github
Enable Cloud Build app in Github Repo. and create cloud build triggers CloudbuildTr
Create Cloud run GO application, simple hello world CloudrunApp
Enable Google container registry GCR
Let's start with CI/CD pipeline through, my environment has TerraForm in version control (GitHub) with Cloud build triggers app on Github and on Google Cloud Platform.
I have another write-up. For senior or advance practitioners move to here
Configuring terraform and google cloud platform will be up to you to decide what is best for your environment but I would highly recommend best practices for both TerraForm and GCP with security in mind. Securing Cloud Run services HERE Also, permissions are important and necessary in order to deploy TerraForm configuration for use of least privilege permissions. I like to keep TerraForm DRY with reusable infrastructure as code so that I can reuse modules. Another key part in this is the template version which I have set to version = "2.1.2".
I have Cloud Build Triggers app within GitHub repo for dev, staging, and prod. This will create changes that I have made using "VScode" and commit to Github. Cloud Build triggers will be notified by branch name of the commit and running either dockerfile, cloudbuild.yaml.
Cloudbuild.yaml configs, I've set the build time to 7200s just to make sure cloud build gives ample time to finish deploying TerraForm configuration. I grab these simple and customizable steps from this tutorial HERE Basic fundamentals of using TerraForm and GCP Cloud Build.
Versions used in this tutorial
~Google provider 3.35.0
~TerraForm v.0.13.0
Once everything is configured, let's start deploying some TerraForm GCP resources
Let's Start
Start by creating a dev folder structure. Here's what's going on, backend.tf is saving remote state to cloud storage bucket. Steps to setup remote state is in the above tutorial, Managing Infrastructure as code. Main.tf is Terraform root dir config, which holds the modules and google provider. Outputs.tf file will pass the output URL from the cloud run modules. Versions.tf is important to keep every TerraForm in sync.
├── README.md
├── cloudbuild.yaml
├── gcloud_build_image
├── environments
│ └── dev
│ ├── backend.tf
│ ├── main.tf
│ ├── outputs.tf
│ └── versions.tf
│ ├── staging
│ ├── prod
├── modules
│ └── services
│ ├── CloudRun
│ │ ├── cloudrun.tf
│ │ ├── outputs.tf
│ │ └── variables.tf
│ ├── cloudEndpoints
│ │ ├── endpoints.tf
│ │ ├── openapi_spec.yml
│ │ └── variables.tf
In the Root dir. we have cloudbuild.yaml and gcloud_build_image, also do a readme.md to explain what the repo and how-to so that other team members can get started.
We have modules/services/ TerraForm resources configurations of cloud run and cloud endpoints. For the sake of this tutorial, I have used hello world with a simple cloud endpoint that are used for the hello GO app. You may need to check configuration for security and if needed use CORS and set IAM permission to the cloud run services.
Moving on to the cloud run module configs I will start with cloud run services. Here we will use the default of cloud run with IAM policy "no-auth" and created variables for the name, location, and a docker image.
# ------------------------------------------------------------------------------
# GCP cloud run application
# ------------------------------------------------------------------------------
resource "google_cloud_run_service" "default" {
name = var.name
location = var.location
template {
spec {
containers {
image = var.dockerimg
}
}
}
traffic {
percent = 100
latest_revision = true
}
autogenerate_revision_name = true
}
data "google_iam_policy" "noauth" {
binding {
role = "roles/run.invoker"
members = [
"allUsers",
]
}
}
resource "google_cloud_run_service_iam_policy" "noauth" {
location = google_cloud_run_service.default.location
project = google_cloud_run_service.default.project
service = google_cloud_run_service.default.name
policy_data = data.google_iam_policy.noauth.policy_data
}
# ------------------------------------------------------------------------------
# variables for Cloud Run
# ------------------------------------------------------------------------------
variable "name" {
description = "variable name for cloud run"
type = string
}
variable "location" {
description = "setting location of service"
default = "us-central1"
}
variable "project" {
description = " setting project name"
type = string
}
variable "dockerimg" {
description = "docker img to be used"
type = string
}
# ------------------------------------------------------------------------------
# outputs for Cloud Run
# ------------------------------------------------------------------------------
output url {
value = google_cloud_run_service.default.status[0].url
}
output urlesp {
value = "${trimprefix(google_cloud_run_service.default.status[0].url, "https://")}"
}
Next, we have cloud endpoints module configs. I've used the data template file to import openapi_spec.yaml into the cloud endpoint config, I also have it in the cloud endpoint module dir. A couple of things going on here, I've created variables for data openapi.yaml config. This will pass in variables from the TerraForm root config into the cloud endpoint module.
# # ------------------------------------------------------------------------------
# # Cloud endpoints
# # ------------------------------------------------------------------------------
data "template_file" "openapi_spec" {
template = "${file("${path.module}/openapi_spec.yml")}"
vars = {
CloudRunES = var.CloudRunESurl ,
HelloAPI = var.ClRnSrvapp
}
}
resource "google_endpoints_service" "api-service" {
service_name = var.CloudRunES2url
project = var.project
openapi_config = data.template_file.openapi_spec.rendered
}
swagger: '2.0'
info:
title: Cloud Endpoints + Cloud Run
description: Sample API on Cloud Endpoints with a Cloud Run backend
version: 1.0.0
host: ${CloudRunES}
schemes:
- https
produces:
- application/json
x-google-backend:
address:${HelloAPI}
protocol: h2
paths:
/hello:
get:
summary: Greet a user
operationId: hello
responses:
'200':
description: A successful response
schema:
type: string
variable "project" {
description = "name of project"
type = string
}
variable "CloudRunESurl" {
type = string
}
variable "ClRnSrvapp" {
type = string
}
Ok so now that we have the module services configured we can jump back to the staging folder and configure the Terraform root config (main.tf)
Cloud Run ESPv2 service will be created at the same time as Cloud Run GO application.
Further guide is at this doc
# # ------------------------------------------------------------------------------
# # Terraform provider
# # --------------------------------------------------------------------------------
provider google {
project = "Project_ID"
region = "var.region"
}
provider "template" {
version = "2.1.2"
}
# ------------------------------------------------------------------------------
# cloud run and cloud endpoints
# ------------------------------------------------------------------------------
module "HelloAPI" {
source = "../../modules/services/CloudRun"
name = "hellotest"
location = "us-central1"
project = "Project_ID"
dockerimg = "gcr.io/${Project_ID}/cloud-run-hello:v2"
}
module "CloudApiESP" {
source = "../../modules/services/CloudRun"
name = "cloudrunesp"
location = "us-central1"
project = "Project_ID"
dockerimg = "gcr.io/${Project_ID}/endpoints-runtime-serverless:cloudapiesp-qutnc7nuq-uc.a.run.app-2020-08-017r0"
}
# ------------------------------------------------------------------------------
# variables for Cloud endpoints
# ------------------------------------------------------------------------------
module "cloudEndpoints" {
source = "../../modules/services/cloudEndpoints"
project = "Project_ID"
CloudRunESurl = "${module.CloudApiESP.urlesp}"
ClRnSrvapp = "${module.HelloAPI.url}"
}
The above config is calling module services cloud run and in that module we are creating name, location, project, and inputting docker image, which is from Google Container Registry. Cloud Run only runs images from GCR. Next, in cloud endpoints module we are creating cloud endpoint and the only configuration we are doing is passing in env variables. What this means is passing the output urls from cloud run module to main config as an output and then to cloud endpoints module env. vars. This is the only way to pass environment variables to main config and then to other module service.
Finally, we're going to create the cloud build yaml config for Continuous deployment. What this does is that cloud build will run steps to create a alpine busy box per say and install Terraform and allows for cloud build to cd into the root folder and modules configurations and be provisioned and deployed. I've also typed in TF_LOG=TRACE which will display TerraForm execution in the background through the cloud build, build log.
Cloudbuild.yaml
# Copyright 2019 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
timeout: 7200s
steps:
# - name: gcr.io/cloud-builders/gcloud
# entrypoint: 'bash'
# args:
# - '-c'
# - |-
# chmod +x gcloud_build_image
# ./gcloud_build_image -s ${cloudrun-esp}-${cloudrun-hash}-uc.a.run.app -c ${config-id} -p ${project-id}
- id: 'branch name'
name: 'alpine'
entrypoint: 'sh'
args:
- '-c'
- |
echo "***********************"
echo "$BRANCH_NAME"
echo "***********************"
#[start tf-init]
- id: 'tf init'
name: 'hashicorp/terraform:0.13.0'
entrypoint: 'sh'
args:
- '-c'
- |
if [ -d "environments/$BRANCH_NAME/" ]; then
cd environments/$BRANCH_NAME
terraform init
else
for dir in environments/*/
do
cd ${dir}
env=${dir%*/}
env=${env*/}
echo ""
echo "*************** TERRAFORM INIT ******************"
echo "******* At environment: ${env} ********"
echo "*************************************************"
terraform init || exit 1
cd ../../
done
fi
# [START tf-plan]
- id: 'tf plan'
name: 'hashicorp/terraform:0.13.0'
entrypoint: 'sh'
args:
- '-c'
- |
if [ -d "environments/$BRANCH_NAME/" ]; then
cd environments/$BRANCH_NAME
terraform plan
else
for dir in environments/*/
do
cd ${dir}
env=${dir%*/}
env=${env*/}
echo ""
echo "*************** TERRAFOM PLAN ******************"
echo "******* At environment: ${env} ********"
echo "*************************************************"
terraform plan || exit 1
cd ../../
cat crash.log
done
fi
# [END tf-plan]
#[START tf-apply]
- id: 'tf apply'
name: 'hashicorp/terraform:0.13.0'
entrypoint: 'sh'
args:
- '-c'
- |
if [ -d "environments/$BRANCH_NAME/" ]; then
cd environments/$BRANCH_NAME
export TF_LOG=TRACE
terraform apply -auto-approve
else
echo "***************************** SKIPPING APPLYING *******************************"
echo "Branch '$BRANCH_NAME' does not represent an oficial environment."
echo "*******************************************************************************"
fi
#[START tf-destroy]
# - id: 'tf destroy'
# name: 'hashicorp/terraform:0.13.0'
# entrypoint: 'sh'
# args:
# - '-c'
# - |
# if [ -d "environments/$BRANCH_NAME/" ]; then
# cd environments/$BRANCH_NAME
# export TF_LOG=TRACE
# terraform destroy -auto-approve
# else
# echo "***************************** SKIPPING APPLYING *******************************"
# echo "Branch '$BRANCH_NAME' does not represent an oficial environment."
# echo "*******************************************************************************"
# fi
#[end tf-destroy]
Once all has been configured, we can then commit to GitHub branch to deploy. We will have two Cloud Run services and one Cloud Endpoints.
Next, we will need to redeploy Cloud Run ESPv2 by rebuilding it, using the gcloud_build_image script provided at the bottom of this tutorial GcloudBuildImage
# - name: gcr.io/cloud-builders/gcloud
# entrypoint: 'bash'
# args:
# - '-c'
# - |-
# chmod +x gcloud_build_image
# ./gcloud_build_image -s ${cloudrun-esp}-${cloudrun-hash}-uc.a.run.app -c ${config-id} -p ${project-id}
We will need to copy Google Container Registry image name so that we can enter that in the Cloud Run ESPv2 Cloud Run service to be redeployed. All we need to do is copy the ESPv2 image name and paste it in the dockimg= ESPv2 image name in the Cloud Run ESPv2 module of TerrForm main.tf.
We should be able to go to the Cloud Endpoint URL is a form of Cloud Run ESPv2 URL /hello.
i.e https://${cloudapiesp-url}-CloudRun-hash.a.run.app/hello
Next, we will change some code on the Cloud Run GO application for continuous integration while rebuilding Cloud Run service. I have a separate GitHub repo. for my Cloud Run application.
Inside of that separate Cloud Run application GitHub Repo. I have a cloudbuild.yaml that is configured with Cloud Build Triggers to that GitHub repo. So everytime I make a commit, I can then run the Continous integration everytime I commit.
steps:
-name: 'gcr.io/cloud-builders/docker'
args: [ 'build', '-t', 'gcr.io/project_id/cloudrun-hello', '.']
-name: gcr.io/cloudbuilders/docker'
args: [ 'push', 'gcr.io/project_id/cloudrun-hello']
-name: 'gcr.io/cloud-builders/gcloud'
args:
[
"run",
"deploy",
"hellotest",
" - image",
"gcr.io/project_id/cloudrun-hello",
" - region",
"us-central1",
" - platform",
"managed",
" - allow-unauthenticated",
]
You could also build the Cloud Run hello application with a new tag then, update the cloud run on the TerraForm config to deploy with the new tag. Cloud Run is not aware of any application update and doesn't automatically pull the latest image from GCR. In the above gcloud cloud build config, you can run a build of docker container image with the new tag
i.e
args: [ 'build', '-t', 'gcr.io/project_id/cloudrun-hello:v2', '.']
Instead of running the docker push command and gcloud deploy command.
Eliminating overhead and complexity.
Then, head back to the module HelloAPI and change the docker image to the version that you set above in the cloudbuild step config.
Let me know of feedback
Latest comments (1)
@pauld first of all, congrats for this article!
do you have this project on Github?