DEV Community

Michael Levan
Michael Levan

Posted on


How To Configure Kube-Prometheus

Whenever you’re implementing a monitoring and observability solution for your environment, a few questions occur like what you’re going to monitor and the specific production environments that need to be monitored.

Another large and important question is how you’re going to monitor the environment.

With Kubernetes and monitoring/observability platforms that go with it, there are a ton of options. It can definitely get a bit overwhelming.

With Kube-Prometheus, the feeling of being overwhelmed should hopefully subside.

In this blog post, you’ll learn all about what Kube-Prometheus is and how to get it up and running.

What Is Kube-Prometheus?

It’s the “easy button”.

The Prometheus Operator itself gives you the ability to install all Prometheus and Grafana features in a declarative way. For example, below is a Kubernetes Manifest to install Alertmanager, which can be installed in a declarative fashion because the Prometheus Operator exists.

kind: Alertmanager
  labels: alert-router main alertmanager kube-prometheus 0.24.0
  name: main
  namespace: monitoring
  nodeSelector: linux
    labels: alert-router main alertmanager kube-prometheus 0.24.0
  replicas: 3
      cpu: 100m
      memory: 100Mi
      cpu: 4m
      memory: 100Mi
    fsGroup: 2000
    runAsNonRoot: true
    runAsUser: 1000
Enter fullscreen mode Exit fullscreen mode

Kube-Prometheus on the other hand installs Alertmanager for you.

Essentially, Kube-Prometheus contains a bunch of Kubernetes Manifests for installing components like Grafana, Alertmanager, Node Exporter, Kube State Metrics, and several other pieces including all of the RBAC permissions and Service Accounts needed to get it up and running.

Instead of installing all of the components manually, you can just use Kube-Prometheus.

Here’s a list of what’s installed:

Image description

Methods Of Installation

Now that you know what Kube-Prometheus is, which is essentially a collection of Kubernetes Manifests to get up and running fast with the Prometheus/Grafana stack, let’s learn a few different installation options.

The first is Helm and the second is a traditional Kubernetes Manifest deployment.

Please note that you don’t have to run both of these. You can run one or the other, or run one, delete one, and then run the next. That way there are no conflicts and you see both processes.


First, add the prometheus-community repo.

helm repo add prometheus-community

Enter fullscreen mode Exit fullscreen mode

Next, ensure that it’s up to date.

helm repo update
Enter fullscreen mode Exit fullscreen mode

When installing Kube-Prometheus, you’ll use the prometheus-community Helm Chart. However, the Kube-Prometheus stack isn’t the only option available when using the prometheus-community Chart. Because of that, ensure that you specify the kube-prometheus-stack option.

helm install kube-prometheus prometheus-community/kube-prometheus-stack
Enter fullscreen mode Exit fullscreen mode

You can find the Helm Chart here:

Image description

Image description

Kubernetes Manifests

First, clone the prometheus-operator Git repository.

git clone
Enter fullscreen mode Exit fullscreen mode

Next, run the configuration for the Kubernetes Manifests.

The following command is used with sleep to avoid any race conditions, which ensures that the proper Service Accounts and RBAC permissions are set up prior to any of the Manifests getting deployed.

kubectl create -f manifests/setup

until kubectl get servicemonitors --all-namespaces ; do date; sleep 1; echo ""; done

kubectl create -f manifests/
Enter fullscreen mode Exit fullscreen mode

Retrieve all Namespaces and you should see one called monitoring.

kubectl get namespaces
Enter fullscreen mode Exit fullscreen mode

After a minute or two, ensure that all of the resources in the monitoring Namespace are up and running.

kubectl get all -n monitoring
Enter fullscreen mode Exit fullscreen mode

Image description

Viewing Grafana

Now that you’ve installed the Kube-Prometheus stack, let’s log into Grafana.

First, retrieve the Grafana service. For the Helm installation, the service name will be kube-prometheus-grafana. For the Manifest installation, the service name will be grafana.

Next, forward the port so you can access the Grafana UI.

For Helm installation:

kubectl port-forward svc/kube-prometheus-grafana :80
Enter fullscreen mode Exit fullscreen mode

For Manifest installation:

kubectl port-forward svc/grafana -n monitoring :3000
Enter fullscreen mode Exit fullscreen mode

The default username/password for Grafana with the Helm Installation is:

  • admin
  • prom-operator.

The default username/password for Grafana with the Manifest installation is:

  • admin
  • admin

Click the Browse button under dashboards.

Image description

Under general (or default), you’ll see a bunch of pre-populated dashboards for you.

Image description

For example, click on the Kubernetes / Computer Resources / Cluster option. You’ll see a Grafana dashboard with information around your cluster.

Image description

What makes Kube-Prometheus so great is that all of the dashboards are populated for you. You don’t have to do much of anything to get the environment up and running out of the box.

You now have a great place to start with Prometheus and Grafana for monitoring/observability of your environment.

Top comments (3)

786gagan profile image

Thanks a lot mate.

mrwormhole profile image
Talha Altınel

manifest setup is breaking my kubectl top nodes and making my metrics server dead, thanks for helm chart guide it really fixed my problems (which are those damned manifests who do the RBAC wrongly)

vaskata84 profile image
Vasil Dobchev

The tutorial is very interesting, but how can I add another configuration in Prometheus, I want to add other devices and show them in the grafana ?

Image description

Join the One Year Club

You can earn this badge by being a registered member of the DEV Community for at least one year. Create an account and get started today.