CircleCI Split Config Orb!

I created a CircleCI Orb named Split Config. This is my private fun coding :)

• https://circleci.com/developer/orbs/orb/bufferings/split-config

Do you want to split your configuration file?

Have you ever thought if you could split the config.yml file of your CircleCI projects? Especially it could be bigger when the repository is a type of MonoRepo which includes multiple services in one repository.

Good news! Split Config Orb is here for you!

Split Config Orb

You can split your config.yml into multiple configuration files. This Orb merges the split configs into one YAML when CircleCI starts and execute a pipeline with the merged configuration.

Example

Let's say you have multiple services in your repo like this:

Well, it's empty but please imagine the source codes of each services are there :)

Each service has a config.yml for the service. For example. the config for service1 is as follows:

version: 2.1

jobs:
  service1-say-hello:
    docker:
      - image: cimg/base:stable
    steps:
      - checkout
      - run:
          name: "Say hello"
          command: "echo Hello, World!1"

workflows:
  service1-say-hello-workflow:
    jobs:
      - common-say-hello
      - service1-say-hello

service2:

version: 2.1

jobs:
  service2-say-hello:
    docker:
      - image: cimg/base:stable
    steps:
      - checkout
      - run:
          name: "Say hello"
          command: "echo Hello, World!2"

workflows:
  service2-say-hello-workflow:
    jobs:
      - common-say-hello
      - service2-say-hello

etc...

The Orb merges these config files and it will be like this:

version: 2.1
jobs:
  common-say-hello:
    docker:
      - image: cimg/base:stable
    steps:
      - checkout
      - run:
          name: Say hello
          command: echo Hello, World! common
  service1-say-hello:
    docker:
      - image: cimg/base:stable
    steps:
      - checkout
      - run:
          name: Say hello
          command: echo Hello, World!1
  service2-say-hello:
    docker:
      - image: cimg/base:stable
    steps:
      - checkout
      - run:
          name: Say hello
          command: echo Hello, World!2
  service3-say-hello:
    docker:
      - image: cimg/base:stable
    steps:
      - checkout
      - run:
          name: Say hello
          command: echo Hello, World!3
workflows:
  service1-say-hello-workflow:
    jobs:
      - common-say-hello
      - service1-say-hello
  service2-say-hello-workflow:
    jobs:
      - common-say-hello
      - service2-say-hello
  service3-say-hello-workflow:
    jobs:
      - common-say-hello
      - service3-say-hello

How to use

It's super simple. You just need to prepare this config.yml

version: 2.1

setup: true

orbs:
  split-config: bufferings/split-config@0.1.0

workflows:
  generate-config:
    jobs:
      - split-config/generate-config:
          find-config-regex: .*/\.circleci/config\.yml

Then the Orb finds the config files according to the regex you specified, and merge them into one YAML file. After that, the Orb executes CircleCI pipeline with the generated YAML file.

It uses the Dynamic Config feature of CircleCI to start a new pipeline. By default, the feature is disabled, so please enable the feature to get started.

Then you will see something like this:

That's all!

Are you interested in how it merges the configs?

How does it merge the configs?

The Split Config Orb uses CUE to merge the YAML files.

• CUE https://cuelang.org/

CUE can import YAML file, convert them into CUE format, and merge them into one CUE config. In addition, it has a feature to export CUE config as YAML.

I used these CUE features to merge your YAML configs. CUE is so powerful but I only use a small part of it. If you want to know more about CUE or how it merges multiple CUE configs, please check CUE specification!

For more details

Please check the README in GitHub:
https://github.com/bufferings/orb-split-config

I wrote some usecases with working examples! Enjoy smaller configuration files