Hey everyone! Today I would like to introduce you to Windmill, a serverless platform enabling you to write scripts in many languages, chain them together, build an UI around them and much, much more. Sounds too good to be true? Well let’s put it to the test by building together a dynamic UI for managing Helm charts in a Kubernetes cluster.
Introduction
There's no doubt that you've found yourself in a situation like this at some point: you’ve been asked to build a quick workflow, where you had to chain a few actions, send a notification somewhere or a piece of data into a system. You have the code ready but then, all of that needs to be packaged and deployed (after tests), and if you’re lucky, you already have all the resources and systems for that part of the process.
But releasing your code into the world is only the initial step in a larger process. Once the code is out there, it needs to be scheduled, executed and observed. This doesn't just happen on its own and you need systems in place to do just that.
This is where serverless Cloud solutions such as “AWS Step Functions” caters to these needs. Step Functions allows you to chain Lambdas, and call upon AWS services with extra logic to your workflow. In essence, it provides a robust platform that allows your code to function as intended, while also offering the necessary tools to monitor and optimize its performance.
See https://aws.amazon.com/fr/blogs/aws/new-aws-step-functions-workflow-studio-a-low-code-visual-tool-for-building-state-machines/ for more info!
Cloud solutions are a great fit indeed, and they’ll probably be good enough for many use cases. But today we’ll talk about an entire platform centered around these workflows, a platform where you could share actions with your coworkers, tie them together in workflows and build UIs around them.
And that’s where Windmill comes in, with three key concepts:
- Scripts, which are re-usable pieces of code that have inputs and outputs
- Flows, which tie scripts together with logic, branching and retries
- Apps, made of dynamic React components which can trigger and display scripts and flows
With these three building blocks, you’ll be able to create powerful and reliable workflows, let’s put it to the test.
I’ve created a local cluster with K3S and installing Windmill could not be simpler with just one chart to configure, which already has sane defaults to get started. For this demo we will also configure workers to passthrough environment variables to our scripts so that they have access to the Kubernetes API server for later.
helm repo add windmill https://windmill-labs.github.io/windmill-helm-charts/
helm show values windmill/windmill > values.yaml
yq -i '.windmill.workerGroups[].extraEnv += {"name": "WHITELIST_ENVS", "value": "KUBERNETES_SERVICE_HOST,KUBERNETES_SERVICE_PORT"}' values.yaml
helm install windmill windmill/windmill --namespace=windmill --create-namespace -f values.yaml --set windmill.appReplicas=1 --set windmill.lsbReplicas=1
Note: see the Chart’s README for all potential values. I recommend managing them finely, with dedicated service accounts for example.
We’ll then watch the pods being created:
$ kubectl get pods
NAME READY STATUS RESTARTS AGE
windmill-app-567858d877-gqp26 0/1 Running 0 32s
windmill-lsp-7bd57bfdfc-bdfg8 1/1 Running 0 32s
windmill-postgresql-0 1/1 Running 0 32s
windmill-workers-default-dc4cc4d76-5wvtl 1/1 Running 0 32s
windmill-workers-native-8df7667f-6xl29 1/1 Running 0 32s 0/1 ContainerCreating 0 20s
While we wait, let’s explore a bit more the architecture of the project and what’s being deployed:
- We have a PostgreSQL instance starting
- An app deployment for the UI and API
- Several workers deployments to execute our scripts
- And an LSP server to provide autocompletions
Once running, you’ll notice an Ingress configuration has also been created, you can adjust the hostname either via values or via the initial configuration steps. Open a browser to your ingress and you’ll be greeted with the default login screen, it will be admin@windmill.dev:changeme:
And after a few settings that you can tweak, you’ll be prompted to change the admin & password email and finally you’ll be able to create a new workspace that will contains all your workflows.
⚠️ There is an opt-out Telemetry option you might not want to miss if testing in a sensitive environment, but it’s also helpful data for the project to grow!
And there you go, we’re in !
Building an Helm UI
To showcase the basic set of features of Windmill, let’s build ourselves a quick Helm UI, that our users could then use to list and rollback releases. On rollback, we’ll also send a notification to Slack.
Listing namespaces
We’ll need a script to list our cluster namespaces, and then the releases it contains.
Click on the “+ Script” button at the top of the window, we’ll select Python as the runtime, give it a good name, and you can click on the editor on the left to get the Settings window out of the way.
We’ll import the kr8s package to interact with the cluster, and when you’ll remove arguments from the main function, you will also see them disappear from the input section! Quite neat.
Let’s write a script to return a list of current namespaces:
import kr8s
def main():
return [ns.name for ns in kr8s.get("namespaces")]
And by pressing CTRL+Enter, execution will start and will error-out:
Which should not be surprising as we did not configure additional permissions to Windmill workers.
Let’s add a quick ClusterRole and ClusterRoleBinding to the windmill user. In practice, we would start adding a custom ServiceAccount for this dedicated worker pool:
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: windmill-k8s
rules:
- apiGroups: [""]
resources: ["namespaces"]
verbs: ["get", "list"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: windmill-k8s
subjects:
- kind: ServiceAccount
name: windmill
namespace: windmill
roleRef:
kind: ClusterRole
name: windmill-k8s
apiGroup: rbac.authorization.k8s.io
Re-running the job, we will see the expected result:
Note the "found cached resolution" debug line. It indicates that Windmill computed and cached all the dependencies necessary to run this script. This will prevent subsequent pip install actions until changes are made.
You can now click on the Deploy button at the top right and CTRL+ENTER on your keyboard to execute the script, where you'll get more details on the duration, memory usage and status.
Listing Helm releases
Since Helm is included by default in the worker's Docker image (thanks for that!), we're able to create a new Bash script:
# shellcheck shell=bash
namespace="$1"
helm list -n $namespace -o json > ./result.json
Our script is accepting an argument “namespace” and we’ll write the result to result.json which will be parsed by Windmill and made available.
We will now have to adjust our RBAC permissions to grant read on the secrets:
kubectl patch clusterrole windmill-k8s --type='json' -p='[{"op": "add", "path": "/rules/-", "value": {"apiGroups": [""], "resources": ["secrets"], "verbs": ["get", "list"]}}]'
And by saving and running our script, we’ll see our list of releases:
Notice how we automatically had an input field for our namespace? Windmill automatically built a quick UI for our script so we're able to run them whenever we want.
Building an Helm UI
With our first scripts ready, we are now able to chain them together to build an UI! Back on the homepage of the workspace, let’s click on the “+ App” button which will open the App editor.
There are many things to present of this editor, but I will refer you to the Youtube playlist made by Windmill authors and the documentation website.
For now, let’s focus on our quick & dirty! Scroll down the list of available components and drag and drop the “Select” component.
Now we need to prepare the list of namespaces for this component. At the bottom of the editor you will find the “Background Runnable” section, click on the + (plus) sign to create a new runnable, click on “Select a script or flow” then on the opened window, select the “Workspace scripts”, then finally, your script listing namespaces.
Note: you might have not missed the “Frontend” language section when creating a new background runnable, you can indeed add extra Javascript code that will be executed in the browser.
The Select input form expect data in a specific format, so we’ll add a transformer, which is just a quick Javascript function to transform our namespaces into what’s expected. When selecting the runnable, you will notice on the right in the settings panel, a “Transformer” section.
Click on the “Add” button and in the new script view, transform the result then click on “Run”:
return result.map((result) => ({label: result, value: result}))
You can inspect the current state of the app and view our reflected data model.
Let’s now plug our Select input. Click on the component, and on its configuration tab, click on the little plug icon next to “Items”, this will allow us to configure the data source for this select. Click on the “result” button in the bg_0 output and the select will be automatically populated!
Next up, we’ll do the same thing with the Helm releases. We create a new background runnable, we select our “List Helm releases” script and this time we’ll notice we’re expected to configure the input. Again, click on the “Connect” plug symbol and we’ll select the result of our select component in the Outputs panel at the left (or you can also write a.result) if you’re in a hurry. Make sure to disable the “Run on start and app refresh” option as we’ll need to have the namespaces first.
Now, every changes to the select value will make Windmill execute again the script with the proper variable, making that data available. Let’s display it with the Table component!
Again, drag & drop the Table component, and select the plug symbol. Select the result of bg_1 which is our list of Helm releases, and boom, you’ve got a table with your releases:
When you select another namespace, the table will refresh with the releases from this namespace. Adjust width and re-organization or add more components if you wish, then you can click on Deploy to get a pretty view that you can share with coworkers:
Great success! You can re-order columns or even hide some if you want by configuring the component.
Wrapping up
And that's it for this first part! I hope I was able to share with you the same excitation I had when I first started using this tool! In the next and final part, we'll look at how we can implement a Flow and add a Rollback button to our table. See you!
Resources
The following resources might be of interest:
- Windmill Youtube channel
- Windmill documentation
- Trevor Sullivan has made a series of videos using Windmill if you want to deep dive
Top comments (0)