DEV Community

Vuong
Vuong

Posted on

Use Regex to match multi Kafka topics Grafana Dashboard

In Grafana Dashboard, when we need to create a new Dashboard to show how many messages has produced by Kafka topics (with Kafka exporter + Prometheus) in a period, we can specify every topic as single metric like this below image

Image description

It works fine for sure! But what if you have more than 2 topics and your topics actually have the same pattern which could be matching by regex. The next time, if any new topic gets created, the dashboard will automatically reflect and show on the Grafana dashboard, then you don't have to do it as the “old” way.

Image description

Here is the query from image above

sum(delta(kafka_topic_partition_current_offset{job="kubernetes-service-endpoints",kubernetes_name="prod-global-kafka-exporter", topic=~"{{regex1}}", topic!~"{{regex2}}"}[1m])) by (topic)
Enter fullscreen mode Exit fullscreen mode
  • {{regex1}}: Replace with a regular expression to match any topic we want to show. Example: prod-myservice.*

  • {{regex2}}: Replace with a regular expression to match any topic we want to avoid showing. Example: prod-myservice.+deprecated.*

A useful web tool to check if the regex could work is https://regex101.com/.

Referenced sources

Happy monitoring!

Discussion (0)