semaphore tutorials (7 Part Series)
Serverless is one of those concepts that in hindsight seems obvious and makes you wonder why no one came up with it earlier.
At its core, it involves breaking up the application into smaller functions, called lambdas, and distributing them in the cloud. Lambdas, by their very nature, scale; this is because they are only run when needed, with resources allocated on-demand. Also, since there are no wasted resources, they tend to be cheaper to run than a full-fledged server or even a container—unless you run them 24x7.
During this post, we will work with a demo app. Step by step, we’ll see how to get it online and how to make part of a continuous delivery process.
The project has a single API Endpoint that simply replies Hello World! to all requests. APIs are the way applications communicate with each other over the web, so mastering them is crucial.
$ cd semaphore-demo-zeit-now $ sem init
The main file is located at
api/hello.js. It has a function that replies with the hello message. Strictly speaking, this is the whole application. You could deploy this file and call it a day. We’ll see that ZEIT Now does a lot of work for us. However, the developer’s life calls for two more things: a development environment and some kind of testing.
With ZEIT Now, we can make a global serverless deployment with just a few keystrokes. The magic resides in their builders which take your existing application code and transforms it into lambda functions. Do you have a lot of static files? No problem. With minification and a smart built-in global CDN, they have you covered.
Internally, Now Lambdas work on top of AWS Lambdas; ZEIT Now also supports other cloud providers (only for caching and routing). As we’ll see next, we won’t have to configure or set up any service in AWS to run our application.
In ZEIT Now, there are no servers to manage and no containers to build. It integrates nicely with any workflow and plays well with CI/CD platforms. With instant, automatic deployments, ZEIT Now is a perfect fit for our microservice app.
Apps in ZEIT Now are immutable, meaning they cannot be changed once deployed. Each newly published version gets a unique deployment URL. As it turns out, this clever approach allows us to roll back to any previous version at any time.
ZEIT Now addresses follow this format:
https://APP_NAME.USERNAME.now.sh: Public-facing URL that points to the latest version.
https://APP_NAME-UUID.now.sh: Deployment URL. UUID is a unique, automatically generated string for each release.
Wouldn’t it be great if we could automate every step of testing and deployment? This is precisely the problem that continuous integration and continuous delivery solve. CI/CD takes care of everthing for us. And it does it as a reproducible, battle-hardened process.
Older continuous integration and delivery platforms, like the stand-alone version of Jenkins, encumber developers an infrastructure to manage. In Semaphore there is no back-end to maintain, no servers to install, or any Java versions to troubleshoot—another point for serverless platforms.
In Semaphore, we define pipelines in a clean, easy-to-read format and do as many Git pushes as needed. Semaphore will silently provision everything to drive the pipeline, at any scale.
In this section, we’ll review how the Continuous Integration pipeline works. The CI pipeline builds and tests the code in the Semaphore environment.
Open the pipeline file located at
.semaphore/semaphore.yml.. We’ll review its elements.
Let’s start with the basics: the name, version, and agent. In the
agent we choose which of the available machine types will run our jobs.
version: v1.0 name: Build and test agent: machine: type: e1-standard-2 os_image: ubuntu1804
These define what to do at each step of the pipeline. Each block can have many jobs. Jobs contain the list of commands to execute. Within a block, jobs run concurrently; each one runs in a fully isolated virtual machine. Once all jobs in a block are done, the next block begins.
The first block downloads and installs all the required packages:
blocks: - name: Install dependencies task: jobs: - name: npm install and cache commands: - checkout - nvm use - cache restore - npm install - cache store
- Checkout clones the code from GitHub.
- Cache is used to share node modules between jobs. It is smart enough to determine which files need to be stored and retrieved.
Since each job lives in an isolated environment, the second block must repeat some of the commands and get the node_modules from the cache. Then we can run the test scripts.
- name: Run tests task: jobs: - name: npm test commands: - checkout - nvm use - cache restore - npm test
Promotions chain pipelines together to build complex workflows. Since they can be triggered by user-defined conditions, they can be used to gracefully manage failures or to make a release.
promotions: - name: Deploy to staging pipeline_file: deploy-staging.yml branch: - name: Deploy to production pipeline_file: deploy-production.yml auto_promote_on: - result: passed branch: - master
We have two branching promotions:
- Deploy to production: automatically started once all tests are green for the master branch.
- Deploy to staging: can be manually initiated from a Semaphore workflow on any branch.
Now that we have all the pieces of the puzzle in place, you will see for yourself the power and convenience of CI/CD.
To connect Semaphore and ZEIT Now we’ll need to get a token from the latter.
- In ZEIT Now go to: Account > Settings > Tokens > Create
The token, being private information, does not belong in the repository. Semaphore has a secure mechanism to handle secrets:
$ sem create secret now -e ZEIT_TOKEN=YOUR_ZEIT_TOKEN
We have two almost identical continuous delivery pipelines: staging and production. They only differ in the deployment name. The difference allows us to have separate URLs for each environment.
blocks: - name: Deploy to production task: secrets: - name: now jobs: - name: Deploy to Zeit Now commands: - checkout - nvm use - npm install now -g - now --token $ZEIT_TOKEN -n semaphore-demo-zeit-now
The deployment itself couldn’t be easier:
- The secret token is decrypted and imported as
- The now cli tool is installed.
- Now does the deployment.
This is where all our hard work pays off:
$ touch any_file $ git add any_file $ git commit -m "start semaphore pipeline" $ git push origin master
In a few seconds, you should have your lambda function deployed to production.
Click on the Promote button under “Deploy to staging” to create the staging site.
Once all the blocks are done, you should have both sites online. Try the APIs:
$ curl -w "\n" https://semaphore-demo-zeit-now-staging.YOUR_NAME.now.sh/api/hello Hello World! $ curl -w "\n" https://semaphore-demo-zeit-now.YOUR_NAME.now.sh/api/hello Hello World!
Nice work! Once you taste the power of serverless architecture and CI/CD, you will want to use it in all your projects. I hope that what you learned today helps you to build better and smarter.
If this article got you interested in serverless, you should check these links:
- Continuous Delivery Patterns with Serverless Applications
- A CI/CD Pipeline for Serverless Cloudflare Workers - Netlify Continuous Deployment
Did you find the post useful? Hit those ❤️ and 🦄, follow me or leave a comment below!
Thanks for reading!