DEV Community

Kunal Kushwaha
Kunal Kushwaha

Posted on

Prevent Kubernetes Misconfigurations from Reaching Production with Datree

The problem

What are some of the challenges that Kubernetes admins might face? Let's say you have a few developers that made changes to K8s configurations files, to deploy their objects, and they are working with CI/CD and they push their configuration file on git repo. Let's say it caused a failure in production. Wouldn't it be cool to figure out these misconfigurations before the production? Comes into picture Datree.

Image description

About Datree

Datree is a CLI tool that supports Kubernetes owners in their roles and it helps by preventing developers from making errors in their Kubernetes configuration files before it reaches production and causing failure. It does so by providing a policy enforcement solution to run automatic checks for rule violations.
It can be used on the command line to run policies against Kubernetes manifests YAML files or Helm charts. You can include Datree's policy check as part of your CI/CD pipeline or run it locally before every commit.

How to sign up

You'll have to sign up in order to access the dashboard consisting your policy checks. Head over to the sign up page and you can choose to either continue to GitHub or Google.
Image description

Follow these steps to get started:
1) Download and install Datree in just one command!
$ curl https://get.datree.io | /bin/bash

2) Scan a Kubernetes file or helm chart directory
$ datree test [k8s-file-path] / helm datree test [chart-dir]

3) You can now click on the link provided in the Summary of the CLI output that says See all rules in the policy to access your dashboard. Here you can also find the cliId in the url itself. This token is what connects the policy checks to your centralized policy to know which policies and rules to run. You can also find your token by going to your profile settings in the dashboard.

Image description

You can also find it via Datree's config via, such as:
$ nano ~/.datree/config.yaml

More information about account tokens can be found here.

How to run

By default, datree offer's a demo Kubernetes configuration file for you to test out the tool. You can find it at ~/.datree/k8s-demo.yaml that you can use to test it out quickly. Let's modify it a little bit.

➜  ~ cat ~/.datree/k8s-demo.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: rss-site
  namespace: test
  labels:
    owner: --
    environment: prod
    app: web
spec:
  replicas: 3
  selector:
    matchLabels:
      app: web
  template:
    metadata:
      namespace: test
      labels:
        app: we
    spec:
      containers:
        - name: front-end
          image: nginx:latest
          readinessProbe:
            tcpSocket:
              port: 8080
            initialDelaySeconds: 5
            periodSeconds: 10
          resources:
            requests:
              memory: "64Mi"
              cpu: "64m"
            limits:
              cpu: "500m"
          ports:
            - containerPort: 80
        - name: rss-reader
          image: datree/nginx@sha256:45b23dee08af5e43a7fea6c4cf9c25ccf269ee113168c19722f87876677c5cb2
          livenessProbe:
            httpGet:
              path: /healthz
              port: 8080
              httpHeaders:
              - name: Custom-Header
                value: Awesome
          readinessProbe:
            tcpSocket:
              port: 8080
            initialDelaySeconds: 5
            periodSeconds: 10
          resources:
            requests:
              cpu: "64m"
              memory: "128Mi"
            limits:
              memory: "128Mi"
          ports:
            - containerPort: 88
Enter fullscreen mode Exit fullscreen mode

You can run the following command in order to test Datree against the pre-compiled demo file:

$ datree test ~/.datree/k8s-demo.yaml

Image description

As you can see, it consists of 5 rules failing in the policy checks. You can also find the history of this in your Datree dashboard.

Image description

Out of the box, Datree offers 30 such rules for you to test out. These rules are spread across categories such as:

  • Containers
  • Workload
  • CronJob
  • Networking
  • Deprecation
  • and more

You can find these rules in the default policy on your dashboard.

Image description

Here, you can either change the error message of various rules, turn some of those on/off, and the changes will be visible in your command line instantly!

Since it's not connected to a cluster, you can also run it offline without needing to setup something like Minikube, Kubeadm, etc.

Schema validation

Datree also performs schema validation checks for your files before running the policy checks. For example, if I make an error in the structure of my YAML file and run a Datree check on it, it's going to give me the following error:

Image description

Here, you can see the order in which these checks are performed. YAML validation -> Kubernetes schema validation -> policy checks

Key features

Datree also allows you to create your own policies. This can useful for different stages of deployment. You can add policies with different rules configured, giving each policy its own use cases. In order to do so, you can head over to your Datree dashboard and click on + Create Policy. Here you can give it a name, and once the policy is created, you can apply rules to it. You can select any or all of the 30 rules for your own policy.

Image description

I am also going to edit the error message to something custom. This is something that is also a huge advantage as compared to using checks provided by kubectl.

Image description

For this example, I'm going to a Kubernetes configuration file that consists of a Deployment and a Service that consists of a database and an api.

 ➜  ~ cat go-demo.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: go-demo-2-db
spec:
  selector:
    matchLabels:
      type: db
      service: go-demo-2
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        type: db
        service: go-demo-2
        vendor: MongoLabs
    spec:
      containers:
      - name: db
        image: mongo:3.3
        ports:
        - containerPort: 28017

---

apiVersion: v1
kind: Service
metadata:
  name: go-demo-2-db
spec:
  ports:
  - port: 27017
  selector:
    type: db
    service: go-demo-2

---

apiVersion: apps/v1
kind: Deployment
metadata:
  name: go-demo-2-api
spec:
  replicas: 5
  selector:
    matchLabels:
      type: api
      service: go-demo-2
  template:
    metadata:
      labels:
        type: api
        service: go-demo-2
        language: go
    spec:
      containers:
      - name: api
        image: vfarcic/go-demo-2:3.0
        env:
        - name: DB
          value: go-demo-2-db
        readinessProbe:
          httpGet:
            path: /demo/hello
            port: 8080
          periodSeconds: 1
        livenessProbe:
          httpGet:
            path: /demo/hello
            port: 8080

--------

apiVersion: v1
kind: Service
metadata:
  name: go-demo-2-api
spec:
  type: NodePort
  ports:
  - port: 8080
  selector:
    type: api
    service: go-demo-2
Enter fullscreen mode Exit fullscreen mode

Now, in order to select this policy instead of the default one when running checks, you can use the -p flag:

$ datree test go-demo.yaml -p My_Policy

Image description

As you can see, this only failed on one of the rules that I added, as compared to 5 rules failing in the default one. You can also notice the custom error message that I have added.

Policy as code

But what if you want to collaborate with other people and share your policies with someone else? This includes following development best practices such as version controlling, automation, collaboration and more. For this, Datree offers something called policy as code. This is a declarative method to represent your policies. When this mode is enabled, the only way to change the policies in your account is by publishing a YAML configuration file. This file is going to contain all your defined policies.

In order to use this feature, you can turn it on from your profile settings and download the policies.yaml file.

Image description

The file is going to look something like this, with the inactive rules commented out:

apiVersion: v1
customRules: null
policies:
  - name: My_Policy
    rules:
      # - identifier: CONTAINERS_MISSING_IMAGE_VALUE_VERSION
      #   messageOnFailure: Incorrect value for key `image` - specify an image version to avoid unpleasant "version surprises" in the future
      # - identifier: CONTAINERS_MISSING_MEMORY_REQUEST_KEY
      #   messageOnFailure: Missing property object `requests.memory` - value should be within the accepted boundaries recommended by the organization
      - identifier: CONTAINERS_MISSING_CPU_REQUEST_KEY
        messageOnFailure: Missing property object `requests.cpu` - value should be within the accepted boundaries recommended by the organization
      # - identifier: CONTAINERS_MISSING_MEMORY_LIMIT_KEY
      #   messageOnFailure: Missing property object `limits.memory` - value should be within the accepted boundaries recommended by the organization
      # - identifier: CONTAINERS_MISSING_CPU_LIMIT_KEY
      #   messageOnFailure: Missing property object `limits.cpu` - value should be within the accepted boundaries recommended by the organization
      # - identifier: INGRESS_INCORRECT_HOST_VALUE_PERMISSIVE
      #   messageOnFailure: Incorrect value for key `host` - specify host instead of using a wildcard character ("*")
      # - identifier: SERVICE_INCORRECT_TYPE_VALUE_NODEPORT
      #   messageOnFailure: Incorrect value for key `type` - `NodePort` will open a port on all nodes where it can be reached by the network external to the cluster
      # - identifier: CRONJOB_INVALID_SCHEDULE_VALUE
      #   messageOnFailure: 'Incorrect value for key `schedule` - the (cron) schedule expressions is not valid and, therefore, will not work as expected'
      # - identifier: WORKLOAD_INVALID_LABELS_VALUE
      #   messageOnFailure: Incorrect value for key(s) under `labels` - the vales syntax is not valid so the Kubernetes engine will not accept it
      # - identifier: WORKLOAD_INCORRECT_RESTARTPOLICY_VALUE_ALWAYS
      #   messageOnFailure: Incorrect value for key `restartPolicy` - any other value than `Always` is not supported by this resource
      # - identifier: HPA_MISSING_MINREPLICAS_KEY
      #   messageOnFailure: Missing property object `minReplicas` - the value should be within the accepted boundaries recommended by the organization
      # - identifier: HPA_MISSING_MAXREPLICAS_KEY
      #   messageOnFailure: Missing property object `maxReplicas` - the value should be within the accepted boundaries recommended by the organization
Enter fullscreen mode Exit fullscreen mode

Working with CI

You can use Datree's policy check in your CI process. This way, every time the CI is triggered, it will also run datree test to verify the Kubernetes configuration files are always configured according to your needs. In order to use Datree, you'll be required your account's token as mentioned above.

Datree supports integrations with:

  • Helm plugin
  • Git hooks
  • CircleCi
  • Travis CI
  • GitHub Actions
  • GitLab CI/CD

You can find examples for these in the public repository.

Resources

Connect with me

Discussion (19)

Collapse
jennifer29ux profile image
JenniferArokiadoss

This is an amazing blog. From the way you gave an situation and an intro about datree, steps, how to run gave a clear understanding of the topic and your video also helped me understand more. As a beginner, this is such a great blog and the resources you shared are really useful.
You're doing an amazing job for students. Feeling blessed that I am a part of your community.

Collapse
fromnibly profile image
Jordan Davidson

I'm still waiting on cuelang to mature a little bit in tooling. It should be able to facilitate most of these tests right in your editor as squiggly lines as a matter of type checking rather than an imperative policy checking.

Collapse
polokghosh53 profile image
Polokghosh53

This is truly great actually like before it makes it to the production line where all the developer will be working on the same stuff and check out to find misconfigurations in the yaml files or wherever in the cluster but it wont be because of Datree.io.
I have used this to check and yeah, this actually makes it easier to locate those problem and rectify them ASAP. People should really know about stuff like this. Really helpful in a long run of working with other devs.👍👍

Collapse
amlandev10 profile image
Amlan 🚀

Thank you very much for this blog ! I've recently started learning about Kubernetes , and now I have an idea about the basics of Datree ! I had heard this term before and checked out its repo on github but didn't have a clear idea about its features and how to actually run it ! Now I've learnt about them due to this article and have read a few sections of its documentation ! So thank you and I'll be sharing what I learnt from this blog on twitter right now !

Collapse
anubhav06 profile image
Anubhav Gupta

Explained in an absolute beginner friendly way.
Was not expecting someone to go in the bottom details so that newbies can also understand.
Although I'm a beginner, but this definitely added a lot to my knowledge.

(Considering my case, I made a discord bot project, and Datree could help me with depreciation of discord js functions/methods which keep on changing to newer versions)

Kudos !

Collapse
mecskyverse profile image
Aakash

Get here from community classroom discord server in greed of Kunal's mentorship . Then I just read the blog and the way of explaination by kunal is too good even though I have started my college journey from this september and don't even know much about kubernetes but understand more about Datree than Kubernetes.

How Datree helps develpor to prevent from errors and the features it provides to add policy rules and edit error message as you want.

Collapse
khushi123456789 profile image
Ksharma

Really insightful blog!! learn the applications of Datree. The best part of using Datree is we can create our own policies and prevention of K8s misconfiguration before it happens. Thanks for sharing this blog with us!!
Great inspiration for us!!

Collapse
yashgangwar7558 profile image
Yash Gangwar

After reading this blog and your Youtube video on Datree I was easily able to setup and run Datree on my local system without any complications. It was soo well explained and beginner friendly. Even I don't have much depth knowledge of Kubernetes but still I was able to understand how this tool is very helpful in preventing Kubernetes misconfigurations.
Thank you kunal bhaiya !!

Collapse
aqsak1 profile image
aqsak1

Hey kunal, it was a great blog. I could get a lot of information about kubernetes and stuff by this. Thankful to you to provide all such amazing tech info to us. Blessed to be connected with you. I have being following you since you started the community. Thanks bro a lot...🎉🎉

Collapse
tabhay56 profile image
abhay thakur • Edited on

hey thanks for this blog First i don't understand what datree is but after reading this blog i can surely say i know what datree do and how it can prevent K8s misconfiguration before it happens

Collapse
rajmishra47 profile image
rajmishra-47

Explained in the simplest possible way i always get overwhelmed by hearing about
Kubernetes ,but with whis article i understood its in best way .Hopeing to see more like this.....

Collapse
sarwesh2003 profile image
Sarwesh2003

Great work kunal

Collapse
ushakirann profile image
usha-Kirann

found useful Thank you for this blog.

Collapse
sarwesh2003 profile image
Sarwesh2003

It's very insightful and informative. Not from kubernetes and DevOps background but surely try this while deploying my side-project application.

Collapse
amit_gaikwad profile image
ameeet

Even I don't know much about kubernetes it was in simplified enough even I got most of it!
Nice one kunal!

Collapse
adarshsingh profile image
Adarsh Singh

Had heard about datree many times in kunal's open source cafe and was somewhat not clear, what it exactly was. But this really cleared all the doubts I had. Totally worth the read.

Collapse
siddhantkhisty profile image
Siddhant Khisty

Interesting blog. I'm gonna be using DaTree once I get familar with K8s. Thanks for this guide on what it does and how to use it. I feel like I have a better understand of using K8s and Datree

Collapse
adeshkhandait profile image
AdeshKhandait

Hey Kunal great blog mannn on problem faced by Developer and its solution using datree. Learned lot more new terminology and problem and how to overcome it. Thanks A lot

Collapse
dilpreetbaath profile image
Dilpreet Baath

Thank you for this blog.