Follow-up of this post. We explained the directory organization and the manifest files naming convention, now let's explore the ansible part.
Ansible was chosen because I knew it a bit since 2015 when I started automating some repetitive tasks to create a Kafka cluster prototype. Ansible is a good candidate because it can talk to HTTP API and it comes with a list of k8s related modules that make easy to query or manage Kubernetes objects (they were rewritten a couple of time before being now really good).
Overview of the Code
I don't plan to review all code, but have a look of key parts of the Ansible code.
- The cluster accept a parameter which is the target cluster, then we load the common variables file then the cluster variable file from
configdirectory
---
- name: "Deploy manifests on cluster {{ target_cluster }}"
connection: local
gather_facts: false
vars:
authorized_cluster: ["dev", "prod"]
hosts: localhost
pre_tasks:
- name: Check if the target_cluster variable was properly set
fail:
msg: "You need to specify the variable target_cluster with one value ({{ authorized_cluster }})"
when: target_cluster not in authorized_cluster
- name: Load common variables
include_vars:
file: configs/common.yml
- name: "Load cluster-scoped variables for {{ target_cluster }}"
include_vars:
file: "config/{{ target_cluster }}.yml"
- for the given cluster, we create a list of all yaml files from the
commonand$clusterdirectories. We will also define some variables than will be used later
---
- name: "Retrieve manifests files from common and {{ cluster }} directories"
find:
paths: ["{{ yaml_dir }}/common", "{{ yaml_dir }}/{{ cluster }}"]
patterns: ['*.yaml', '*.yml']
file_type: file
recurse: true
register: manifests_list
- name: "Define variable from manifests filename"
vars:
filename: "{{ item.path | basename }}"
kind: "{{ item.path | basename | regex_replace('.*_(.*)_.*_.*_.*\\.ya?ml', '\\1') }}"
name: "{{ item.path | basename | regex_replace('.*_.*_(.*)_.*_.*\\.ya?ml', '\\1') }}"
namespace: "{{ item.path | basename | regex_replace('.*_.*_.*_(.*)_.*\\.ya?ml', '\\1') }}"
state: "{{ item.path | basename | regex_replace('.*_.*_.*_.*_(.*)\\.ya?ml', '\\1') }}"
set_fact:
manifests_information: "{{ manifests_information + [{ 'filepath': item.path, 'filename': filename, 'kind': kind, 'name': name, 'namespace': namespace, 'state': state }] }}"
loop: "{{ manifests_list.files | sort(attribute='path') }}"
loop_control:
label: "{{ filename }}"
- Now we check each file honors the filename convention else we throw an error.
- name: "Check files match the naming convention"
fail:
msg: >
The file {{ item.filepath }} does not observe the manifest naming convention.
when: item.filename == item.kind or
item.filename == item.name or
item.filename == item.namespace or
item.state not in [ "present", "absent" ]
with_items:
- "{{ manifests_information }}"
loop_control:
label: "{{ item.filepath }}"
- for each file, call the
k8smodule, the path of of the file is passed to argument using the lookup plugin template, so the manifest can have logic and variables inside.
---
- name: "Deploy {{ item.kind }} {{ item.name }} on namespace {{ item.namespace }} to {{ item.state }}"
k8s:
host: "https://xxxxxx:6443"
api_key: "{{ token }}"
state: "{{ item.state }}"
definition: "{{ lookup ('template', item.filepath) }}"
For the code we're done, for a stripped down version it does not require much than that to work.
Coding your Manifests
One of the feature this small playbook permits is to have simple manifest yaml file like this
---
apiVersion: project.openshift.io/v1
kind: Project
metadata:
name: "{{ cluster-admin-ns }}"
...
but you can also using feature like Ansible variables and Jinja templating together.
For instance, I had to put a config file into a secret, which require to have the content base64 encoded. It's convenient when you can do that automatically.
- In this example, the content for the key
fluent.confcome from a filefluent.conf.j2loaded by thetemplatelookup (so the variable are replaced) then passed to the filterb64encode.
---
apiVersion: v1
data:
fluent.conf: "{{ lookup('template', 'fluent.conf.j2') | b64encode }}"
kind: secret
metadata:
name: fluent-forwarder-secret
namespace: logging
...
with the configuration file treated as a template
# fluent.conf.j2
# As this file is interpreted as a ansible template, you can add
# jinja code inside to put some logic.
<source>
@type forward
port 24224
tag logs.openshift
</source>
<match **>
# Send all types to Splunk
@type splunk_hec
hec_host {{ splunk.host }}
hec_port {{ splunk.port }}
hec_token {{ splunk.token }}
index {{ splunk.index }}
</match>
so it gives once loaded something like
apiVersion: v1
data:
fluent.conf: IyBmbHVlbnQuY29uZi5qMgojIEFzIHRoaXMgZmlsZSBpcyBpbnRlcnByZXRlZCBhcyBhIGFuc2libGUgdGVtcGxhdGUsIHlvdSBjYW4gYWRkCiMgamluamEgY29kZSBpbnNpZGUgdG8gcHV0IHNvbWUgbG9naWMuCgo8c291cmNlPgogIEB0eXBlICBmb3J3YXJkCiAgcG9ydCAgMjQyMjQKICB0YWcgbG9ncy5vcGVuc2hpZnQKPC9zb3VyY2U+Cgo8bWF0Y2ggKio+CiAgIyBTZW5kIGFsbCB0eXBlcyB0byBTcGx1bmsKICBAdHlwZSBzcGx1bmtfaGVjCiAgaGVjX2hvc3Qgc3BsdW5rLmRvbS50bGQKICBoZWNfcG9ydCA4MDg5CiAgaGVjX3Rva2VuIDM1NDM0My0zNDUzMzMtMTUyMzMKICBpbmRleCBmYWtlLWluZGV4CjwvbWF0Y2g+
kind: Secret
... Just what expect the secret API.
So I think we're good for now, I just wanted to give you an idea of what it is possible to do. This is a skeleton that can be enhanced with more check and feature.
Perhaps I'll come back with new idea.
Top comments (2)
Good job!
Sometimes it might be difficult to code a manifest from an already existing resource in the cluster (for instance, patching one field of a cluster operator).
I'm wondering to what extent Kustomize can help and if it could be integrated with your solution.
Cheers
Thanks Bro !