DEV Community

Adam Gardner
Adam Gardner

Posted on • Updated on

Hands On: Pull metrics into Kubernetes from anywhere and treat them generically with the Keptn Metrics Server

Metrics, Metrics, Everywhere

Metrics are everywhere because tool sprawl is real. If you work in anything beyond a startup, you'll most likely have multiple metric-storing backends. Multiple Prometheus instances, endless vendor solutions, homegrown databases etc.

What if...

What if you could just say "get me a metric called foo" and magically, it appeared?

That's precisely what the Keptn Metrics Server allows.

How the Keptn Metrics Server Works

Install Prometheus

The first thing you'll need, of course, is at least one backend to store metrics. So install Prometheus now:

helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm repo update
helm install kube-prom-stack prometheus-community/prometheus
Enter fullscreen mode Exit fullscreen mode

Install Keptn

helm repo add keptn https://charts.lifecycle.keptn.sh
helm repo update
helm install keptn keptn/keptn -n keptn-system --create-namespace --wait
Enter fullscreen mode Exit fullscreen mode

Expose Prometheus and Get an Existing Metric

Now expose the Prometheus UI and get the name of a metric you would like to use.

Get the names of the services. Look for one called prometheus-server running on port 80 (that's the Prometheus user interface):

kubectl get svc
Enter fullscreen mode Exit fullscreen mode

Port forward to access the UI on port 8080:

kubectl -n default port-forward svc/prometheus-server 8080:80
Enter fullscreen mode Exit fullscreen mode

Go to http://localhost:8080 and in the search box, type prometheus_.

Choose a metric which has a non-zero value. For example:

prometheus_http_requests_total{code="200", handler="/-/ready", instance="localhost:9090", job="prometheus"}
Enter fullscreen mode Exit fullscreen mode

Make a note of that PromQL. You'll need it later.

Create KeptnMetricsProvider

A KeptnMetricsProvider is how you define where metrics are coming from. One KeptnMetricsProvider is created for each "backend".

So create one now. I'm deciding to call it local-prometheus:

keptnmetricsprovider.yml

---
apiVersion: metrics.keptn.sh/v1alpha3
kind: KeptnMetricsProvider
metadata:
  name: local-prometheus
  namespace: default
spec:
  type: prometheus
  targetServer: "http://prometheus-server.default.svc.cluster.local:80"
Enter fullscreen mode Exit fullscreen mode

Apply it:

kubectl apply -f keptnmetricsprovider.yml
Enter fullscreen mode Exit fullscreen mode

See all KeptnMetricsProviders:

kubectl get keptnmetricsproviders
Enter fullscreen mode Exit fullscreen mode

Create a KeptnMetric

A KeptnMetric is created to define what you want to retrieve, and from where (which leverages the KeptnMetricsProvider) and how often the metric should be retrieved (fetchIntervalSeconds).

The KeptnMetricsProvider and KeptnMetric must be in the same namespace.

keptnmetric.yml

---
apiVersion: metrics.keptn.sh/v1alpha3
kind: KeptnMetric
metadata:
  name: prometheus-http-requests-total
  namespace: default
spec:
  provider:
    name: local-prometheus
  query: 'prometheus_http_requests_total{code="200", handler="/-/ready", instance="localhost:9090", job="prometheus"}'
  fetchIntervalSeconds: 10
Enter fullscreen mode Exit fullscreen mode

Show all Keptn Metrics with:

kubectl -n default get keptnmetrics
Enter fullscreen mode Exit fullscreen mode

That's it. Use any metric generically regardless of backend

All of your metrics are now available on-cluster to use for whatever you need. Horizontal Pod Autoscaling, FinOps etc. etc.

Top comments (0)