You've probably heard of CI/CD or DevOps. Maybe you have read my Toolchain article or listened to the Tech Jam podcast I co-host.
Build/Test/Deploy your Go App using IBM Cloud Toolchain
Liam Conroy Hampton ・ Sep 24 '20 ・ 10 min read
Wherever you have heard it, it is a hot topic and there are so many elements to it. In this blog I will show you how I deploy two microservices into two different clouds. However, I have a plot twist. I am only using 1 pipeline. This is called multi-cloud deployment, a method that has many benefits from cost savings to security and reliability. This is becoming a more popular method of cloud deployment and this article just scratches the surface.
The 3 takeaways from this article:
- Learn something new about CI/CD
- Learn how to write a Tekton pipeline
- Understand how easy it is to deploy into more than one cloud
For context, I have written a client app in Golang and a server app in Node.js. The Go app will submit/POST a piece of data through a form input box and the Node app will receive this and display it. Pretty simple but it does the job.
If you also want to do this, here are some pre-reqs:
- IBM Cloud Account
- DigitalOcean Cloud Account
- GitHub Account
- 2 microservices (ensure they are both in GitHub or equivalent). Feel free to use mine and just sub in the values throughout the demo
Building a TekTon pipeline
TekTon is purpose build for Knative applications. It handles containers and deployment into Knative environments seemlessly. Whereas you would previously have a Jenkinsfile or travis.yaml in the root of your project you dont neceessarily need this in TekTon (at least for this demo anyway).
A TekTon pipeline provides new resources through Custom Resource Definitions (CRD's). These are:
A TekTon pipeline can consist of many tasks and within the tasks can be many steps. This will become clearer as we go along.
The most fundamental part to always remember is the pipeline is decoupled and the building blocks are isolated containers.
Step 1 - Set up an IBM and DigitalOcean Cloud Accounts and acquire access keys
Just follow these links to sign up (they are free)
IBM token creation
- Sign into the IBM Cloud Managemant Console
- Manage -> Access (IAM) -> API keys ->
Create an IBM Cloud API key
DigitalOcean token creation
- Sing into the DigitalOcean Management Console
- Select API from the list on the left
- Select the
Generate New Token
Keep both of these api keys safe and ensure you know which is which. You will not be able to see them again.
Step 2 - Write the pipeline yaml
File 1 -
In this file you can see the main flow of the pipeline. It will firstly execute
pipeline-ibm-cloud-install-task and then afterwards it will execute
apiVersion: tekton.dev/v1beta1 kind: Pipeline metadata: name: deployment-pipeline spec: params: - name: ibmcloudapikey description: The IBM Cloud API Key - name: doauth description: The DigitalOcean API Key - name: file description: The node app repository contents to download - name: uuid description: A unique identifier tasks: - name: pipeline-ibm-cloud-install-task taskRef: name: ibm-cloud-install-task params: - name: ibmcloudapikey value: $(params.ibmcloudapikey) - name: file value: $(params.file) - name: uuid value: $(params.uuid) - name: pipeline-do-cloud-install-task runAfter: pipeline-ibm-cloud-install-task taskRef: name: do-cloud-install-task params: - name: doauth value: $(params.doauth) - name: uuid value: $(params.uuid)
File 2 -
This is a
Task and it has a single step. The primary image declared is
ibmcom/ibm-cloud-developer-tools-amd64:1.2.3 and it has the
ibmcloud cli pre-installed and set up. First it downloads a zip of my project from GitHub. Secondly, it uses the ibmcloud cli to authenticate my IBM Cloud account and pushed my project up to CloudFoundry.
apiVersion: tekton.dev/v1beta1 kind: Task metadata: name: ibm-cloud-install-task spec: params: - name: ibmcloudapikey type: string - name: file type: string - name: uuid type: string steps: - name: login-and-deploy-function image: ibmcom/ibm-cloud-developer-tools-amd64:1.2.3 script: | #!/usr/bin/env sh apk add zip mkdir -p ./application wget $(params.file) -O nodeapp.zip unzip ./nodeapp.zip -d ./application rm nodeapp.zip echo $(params.uuid) cd application/microservice-node-demo-main ibmcloud login -a cloud.ibm.com -r eu-gb -g Default -apikey $(params.ibmcloudapikey) ibmcloud cf install ibmcloud target --cf ibmcloud cf push liams-node-app -m 128M echo PUSHED TO IBM CLOUD CF
Be sure to change the name of your app by changing the line
ibmcloud cf push.
File 3 -
Task is using a vanilla Alpine image, installing the DigitalOcean CLI (
doctl), creating an
app spec (a deployment
.yaml file DigitalOcean expects) and then finally deploying the app. The
app spec consists of the constraints I wish to impose on my application upon deployment. As you can see, I have hard coded an environment variable (
APPROUTE) for my application to use to connect to my other app deployed in IBM Cloud. This could be subsituted with a parameter but this is just a basic demo.
In the code below, make sure you copy your own values into the bits that need changing (
APPROUTE value) and tailor it to your applications needs. DigitalOcean App Spec.
If you are using my repositories just sub in the values.
apiVersion: tekton.dev/v1beta1 kind: Task metadata: name: do-cloud-install-task spec: params: - name: doauth type: string - name: uuid type: string steps: - name: login-and-deploy-function image: alpine:3.13.0 script: | #!/usr/bin/env sh echo pipeline-uuid-$(params.uuid) apk add wget wget https://github.com/digitalocean/doctl/releases/download/v1.54.0/doctl-1.54.0-linux-amd64.tar.gz tar xf doctl-1.54.0-linux-amd64.tar.gz mv doctl /usr/local/bin mkdir -p ./appspec cd appspec/ touch client-go-app.yaml ls -la cat >> client-go-app.yaml <<EOF name: client-go-app services: - name: go-app git: repo_clone_url: <your-github-project-url-to-deploy-into-digital-ocean> branch: do-deploy build_command: go build -o goapp run_command: ./goapp envs: - key: APPROUTE value: https://<your-ibmcloud-app-name>.eu-gb.mybluemix.net instance_count: 1 EOF pwd doctl auth init -t $(params.doauth) doctl apps create --spec client-go-app.yaml echo App deployed to DigitalOcean
As it stands this will not work and you'll find out why shortly
Once these files have been created, create a new repository in GitHub and push these files into it.
Step 3 - Build your Toolchain in IBM Cloud
- Login to your IBM Cloud account
- Navigate to the DevOps section in IBM Cloud using the navigation menu on the left (its about half way down the list)
- Select your region (I chose London) and then click on
- Scroll down to the bottom and select "Build your own toolchain"
- Give it a name and leave the region and resource group as the defaults
You have now created a foundation to build on. At this point your toolchain will be empty, so lets fill it.
- In the top right, click on the button
Add tooland search for
GitHub. This will be used to access the microservices in GitHub or an equivalent server.
Fill out the details:
- Server - Where the projects lives (default is GitHub).
- Repository type - Change this to
Existingsince your projects already exists.
- Repository URL - This is the HTTPS project URL (not the
- Integration Owner - This is your GitHub username. You may be propted to allow Toolchain access to your GitHub profile. Once connected, it will auto-populte.
Ignore the checkboxes
Enable GitHub Issues and
Track deployment of code changes as we do not need them for this demo.
Repeat these steps and add a new GitHub
tool in the Toolchain for each GitHub repository. 1 for each microservice and 1 for the Tekton pipeline repository.
Once you have done this, add one final tool. Thats right, FINAL tool! Click
Add tool and search for
Delivery Pipeline. This is the workhorse in this Toolchain.
Select it and proceed to fill out the following:
- Pipeline name - Give it a unique name
- Pipeline type - Here you have 2 options.
Classic, which provides a graphical interface and allows you to deploy your applications into CloudFoundry and
Tekton, a cloud native CI/CD pipeline tool. Select
Tektonfrom the list.
Now click on
If you get an error box saying
"Continuous Delivery service required"dont panic. In the main IBM Cloud search bar, search for continuous delivery and select "Continuous Delivery" - You only need to do this once!
Its very simple to set up this service. Make sure the region is set to the same as your toolchain (mine is London), give it a sensible name such as "toolchain delivery service" and click
Create. Now head back to your Toolchain and the error should be gone.
Step 4 - Configure the Tekton pipeline
Click on the
Delivery Pipeline tool and this will take you to the
TektonPipeline Configuration dashboard.
On the left side of the screen you will see a list. These are as follows:
Definitions - This specifies the pipeline code to be run upon a trigger. We created 3 Tekton
.yaml files in Step 2. Click on
Add and you will be given a popout. For the
Repository select the Tekton pipeline repository that has the
.yaml files we created earlier in. Chose the
main branch in the next box and leave
Path blank if you have left the files in the root directory of the project repository. Click
Add. Once the code has loaded in, click on
Validate, wait for this to finish and then click
Worker - This is essentially how your pipeline will run. You can have dedicated private workers or ones that from a shared pool of resources. We will just use a shared resource so select
IBM Managed workers (Tekton Pipelines vx.xx.x) in LONDON and click
Triggers - These specify what happens when an event occurs. More often than not, they will be event triggers from a webhook on GitHub, such as a pull request or a new commit or merge. However in this blog we will be using a manual trigger. We'll set this up shortly.
Environment properties - Key:Value pairs that can be used from within your pipeline. These are referenced as
$(PARAMS) in the
.yaml files. In here we need to
Add 4 evironment properties for this pipeine to run correctly and they are:
|doauth||Secure||DigitalOcean api key|
|file||Text||.zip url of 1 project repsitory (mine is my node project)|
|ibmcloudapikey||Secure||IBM Cloud api key|
|uuid||Text||Unique identifier e.g 1234|
.zipurl = url + /archive/main.zip for example:
Other settings - Use these settings for more granular control over builds. For this demo we will not be touching these.
Now everything has been set up. The pipeline code has been imported, the workers have been set and the evironment properties have been set. Seems like it is good to go.. right? Try and click on
Run Pipeline in the top right of your screen.
PSYCH! Got cha! This will not work because we have not got a trigger. It doesn't know what to do when that button is clicked so we need to tell it.
Like I mentioned, it would usually be a trigger from an event that happens on a repository, such as a pull request or merge but we will create a manual trigger.
We need to revisit our pipeline code repository and add some more files. I have named these
hello-xxx.yaml to give a sense I am envoking a new conversation with it but you can call them whatever you like, just make sure to change the corrosponding code.
File 4 -
apiVersion: tekton.dev/v1beta1 kind: EventListener metadata: name: hello-listener spec: triggers: - binding: name: hello-trigger-binding template: name: hello-trigger-template
File 5 -
apiVersion: tekton.dev/v1beta1 kind: TriggerBinding metadata: name: hello-trigger-binding spec: params: - name: ibmcloudapikey value: $(event.ibmcloudapikey) - name: doauth value: $(event.doauth) - name: file value: $(event.file) - name: uuid value: $(event.uuid)
File 6 -
apiVersion: tekton.dev/v1beta1 kind: TriggerTemplate metadata: name: hello-trigger-template spec: params: - name: ibmcloudapikey description: The IBM Cloud API Key - name: doauth description: The DigitalOcean API Key - name: file description: The node app repository to be downloaded - name: uuid description: A unique identifier resourcetemplates: - apiVersion: tekton.dev/v1beta1 kind: PipelineRun metadata: name: pipelinerun-$(params.uuid) spec: pipelineRef: name: deployment-pipeline params: - name: ibmcloudapikey value: $(params.ibmcloudapikey) - name: doauth value: $(params.doauth) - name: file value: $(params.file) - name: uuid value: $(params.uuid)
Push these new files into your main branch on GitHub and inside your pipeline dashboard in IBM Cloud, go to
Definitions -> click the 3 dots on the repository already imported ->
Edit -> click on the
Update button ->
Head to the
Triggers page and then click
Add trigger. Select
Manual. You should only have 1 EventListener to choose from and it should be called
hello-listener. On the
Worker box, select the option
Inherit from Pipeline Configuration as this will use the shared pool or works, just like we set up earlier. Click
Wahoo! You have finished setting up your Tekton pipeline!
Now all thats left to do is run it.. click
Run Pipeline, enter a
uuid value for the pipeline run and click
Run. Watch the magic happen 🎉
My challenge for you
Create a new
Task to clean up each deployment at the end of the pipeline run and let me know how you got on 😄
If you have any questions or want to see more content like this, drop me a line!
Top comments (0)