Using CircleCI’s orbs is a great way to share CI/CD configuration across projects. Public orbs work well for wide adoption, but private orbs have been helpful for organizations needing to share common internal configuration in a secure, non-public way. Private orbs work only within the organization that publishes them.
We recently opened up private orbs access to all CircleCI customers, including those on the Free plan. This new access opens up a number of new possibilities for your pipelines.
In this article you will learn about private orbs and how to use them for your own organization. We will be using the sample orb I built, which contains a command and a job that lets you trigger CircleCI pipelines in other projects using cURL. You can find the source code of the project in this repository.
Prerequisites
This article assumes:
- Working knowledge of CircleCI
- Understanding of how pipelines work
- CircleCI CLI installed and configured locally on your machine
- You are using either a project organization or your personal organization on GitHub
Use cases for private orbs
Like orbs in general, private orbs are useful for sharing steps and tasks that are common to multiple projects.
Many organizations have their own frameworks and internal tools used across many teams. With private orbs they can share those tools easily, without opening it up to the wider world. Some use cases for private orbs include deployment to your proprietary Kubernetes clusters or other infrastructure, dealing with custom hardware, all the way to logging or common admin tasks.
Initial setup
We will use the CircleCI CLI for this task. With the CLI you will create a new orb, and register it with CircleCI so it can be referenced by other CircleCI pipelines. The CLI allows you to release new versions of your orb.
The whole orb creation process is documented in the CircleCI docs. This article will serve as a quick recap of it and explanation of the steps involved.
Create an empty git repository and note its path. I created mine under my zmarkan-demos
organization, and named it api-requester-orb-sample
, so the path is: git@github.com:zmarkan-demos/api-requester-orb-sample.git
.
Next, create the actual orb with the CLI’s circleci orb init
command, passing in a local path where you want the orb source to be located. This directory must be new; do not use one that already exists. In my case, this was:
circleci orb init ~/github/api-requester-orb-sample --private
The main difference between public and private orbs is the --private
flag at the end.
The CLI asks you to either follow the guided setup, which is what I did, or to do it yourself. I recommend using the automated process is recommended as it will set up everything in the directory with an empty orb template, as well as link it to your repository, create a namespace for your orb, and create a CircleCI project for the new orb, just make sure to copy the git URL from the repository you created earlier.
Finally you need to push the local code to the remote repository -
cd ~/github/api-requester-orb-sample
git push origin master
Note: All orbs exist in a namespace within your organization. This is the same for any runner agents you have registered already.
You can also review your new private orbs in your organization settings, where they are clearly labeled as private under the type
section.
Developing an orb
The orb development process depends on whether you opt for the automated guided process or follow your own. As with the setup section, the CircleCI docs cover both scenarios.
If you have followed the quick setup route, then the CLI will have generated the whole project for you from template. It has samples for a command, job, executor, and a test for it.
You can remove any part of the orb that you do not need. In my case I removed the scripts and executors, and kept the commands and jobs. Orbs are just a collection of CircleCI configuration scripts that get resolved and bundled when you run a pipeline.
We created 2 scripts:
- A
commands/trigger-pipeline.yml
command - A
jobs/trigger-circleci-pipeline.yml
job
The job wraps the command and executes in the base Docker executor.
Using the trigger-pipeline command
The trigger-pipeline
command has 4 parameters, and a single step, as shown in this code sample:
description: >
This command triggers CircleCI pipeline specified.
The job execution environment must have cURL utility available
parameters:
repo-name:
type: string
description: "Repo name to trigger pipeline for"
branch-name:
type: string
default: main
description: "Branch name to trigger"
trigger-params-json:
type: string
default: "{}"
description: Pipeline trigger parameters in JSON format
circle-token-env-var:
type: env_var_name
default: CIRCLE_TOKEN
description: Environment variable containing your CircleCI API Key. Default is $CIRCLECI_API_KEY
steps:
- run:
name: Run cURL request
command: |
curl --request POST \
--url "https://circleci.com/api/v2/project/gh/${CIRCLE_PROJECT_USERNAME}/<< parameters.repo-name >>/pipeline" \
--header "Circle-Token: ${<< parameters.circle-token-env-var >>}" \
--header "content-type: application/json" \
--data '{"branch":"<< parameters.branch-name >>","parameters":<< parameters.trigger-params-json >>}"'
Unlike a full CircleCI pipeline config, this one does not require a version
stanza. With only a single command source, it doesn’t have jobs
or workflows
either. All 4 parameters are also specific to this command, unlike the pipeline parameters available in the full CircleCI config.
The curl
command in the run step hits the CircleCI API to trigger another pipeline. It is described in detail in this blog post.
Using the trigger-circleci-pipeline job
Writing a job that consumes the command we just created is straightforward. Within the same orb, you can use that command as if it were a local command you had defined in the config. In our example in jobs/trigger-circleci-pipeline.yml
it looks like this:
description: >
Job that trigger CircleCI pipeline in another project with the payload specified
docker:
- image: cimg/base:2021.12
parameters:
repo-name:
type: string
description: "Repo name to trigger pipeline for"
branch-name:
type: string
default: main
description: "Branch name to trigger"
trigger-params-json:
type: string
default: "{}"
description: Pipeline trigger parameters in JSON format
circle-token-env-var:
type: env_var_name
default: CIRCLE_TOKEN
description: Environment variable containing your CircleCI API Key. Default is $CIRCLECI_API_KEY
steps:
- trigger-pipeline:
repo-name: << parameters.repo-name >>
branch-name: << parameters.branch-name >>
trigger-params-json: << parameters.trigger-params-json >>
circle-token-env-var: << parameters.circle-token-env-var >>
This command has the parameters, and as with all other jobs in CircleCI we need the executor: docker
in our case. The steps
stanza contains a single step, invoking our trigger-pipeline
command from before, with any parameters we set.
Testing and deploying the orb
To build, test, and deploy our orb, the orb template project uses CircleCI itself. Within the config in .circleci/config.yml
most everything is pre-made for you. All you need to do is to write tests for it. By default, it contains a job named integration-test-1
, which you can modify to use your own orb. Here is a snippet that invokes the trigger-pipeline
command:
jobs:
integration-test-1:
docker:
- image: cimg/base:stable
steps:
- checkout
- api-requester-orb-sample/trigger-pipeline:
repo-name: pinging-me-softly
branch-name: main
trigger-params-json: '{"image-name": "cimg/base:2021.12"}'
To publish a release version you need to:
- Change from the default
alpha
branch to either themaster
ormain
branch - Include a
[semver:patch|minor|major]
message in the commit
Conclusion
In this article you have learned about building private orbs for CircleCI, which are now available to all CircleCI customers. Private orbs allow you to have shared CI/CD pipeline configuration across multiple projects. These private orbs can be used by any team member in your organization, without the risk of exposing your config to the public.
The process as a whole is almost identical for publishing public orbs. Just add the --private
flag in the circleci orb init
command.
To read more about orb development check out the docs pages on authoring orbs.
To read about testing and verifying bash scripts in your orbs using BATS and ShellCheck, check out this docs page on orb testing methodologies.
If you have any questions or suggestions about this article, or ideas for future articles and guides, reach out to me on Twitter - @zmarkan or email me.
Top comments (0)