DEV Community

MD Taseen Khan
MD Taseen Khan

Posted on • Originally published at reactjsexample.com on

A monitoring tool built to visualize metrics from local kafka clusters

A monitoring tool built to visualize metrics from local kafka clusters

A monitoring tool built to visualize metrics from local kafka clusters

KlusterFunk is a monitoring tool, built to visualize metrics from local kafka clusters, showing you a real-time, live-updating graph of those metrics.

You can find and use our application at https://klusterfunk-b05ffb62bc07.herokuapp.com/

Overview

This is a tool meant for developers that are familiar with Kafka and how to export metrics. To use this tool we assume you have:

  1. Implemented Prometheus’ JMX exporter on your running Kafka cluster(s).
  2. Set up a Prometheus instance including a yaml config file that is pulling metrics from the port where JMX exporter metrics are being exposed.
  3. Port forward your Prometheus port so you can input the link in our app.

If you have not yet set up these tools for your clusters, follow the steps in Setup

App in Action

A monitoring tool built to visualize metrics from local kafka clusters

Features

Feature Status
Login authorization using JWT tokens
Prometheus API
Build in functionality to have users simply input kafka cluster URIs and link up metrics on the backend
Allow users to choose from list of metrics they would like to see or even input metrics they want to see
Switch between Kafka clusters
Dark Mode
More styling using Tailwind 🙏🏻
Save in database location of kafka clusters and prometheus address 🙏🏻
  • ✅ = Ready to use
  • ⏳ = In progress
  • 🙏🏻 = Looking for contributors

Setup

To setup JMX exporter

  • Build exporter:
    git clone https://github.com/prometheus/jmx_exporter.git
    cd jmx_exporter
    mvn package
Enter fullscreen mode Exit fullscreen mode
  • Start zookeeper:
  /usr/local/opt/kafka/bin/zookeeper-server-start /usr/local/etc/zookeeper/zoo.cfg
Enter fullscreen mode Exit fullscreen mode
  • Setup JMX exporter to run on Kafka (run from root) \
export EXTRA_ARGS="-Dcom.sun.management.jmxremote \
 -Dcom.sun.management.jmxremote.authenticate=false \
 -Dcom.sun.management.jmxremote.ssl=false \
 -Djava.util.logging.config.file=logging.properties \
 -javaagent:/Users/<username>/jmx\_exporter/jmx\_prometheus\_javaagent/target/jmx\_prometheus\_javaagent-0.20.1-SNAPSHOT.jar=8081:/Users/<username>/jmx\_exporter/example\_configs/kafka-2\_0\_0.yml"
Enter fullscreen mode Exit fullscreen mode
  • Start kafka
/usr/local/opt/kafka/bin/kafka-server-start /usr/local/etc/kafka/server.properties
Enter fullscreen mode Exit fullscreen mode

Localhost:8081 should now be displaying JMX metrics

  • Run Prometheus and point it to scrape at port 8081 (or whatever port you configured the JMX exporter )

Contributing

Feel free to GitHub issues to let us know what features you want and what you’d like to see next from the project!

If you would like to work on the open source code, please do feel free to submit a pull request! Make sure you’re following Javascript ES6 syntax and modularize your code as much as possible.

To get started, first fork the repo to your personal github repos, then run the following commands:

git clone https://github.com/oslabs-beta/KlusterFunk.git

npm install

npm run dev
Enter fullscreen mode Exit fullscreen mode

Stack

Apache Kafka, JMX Exporter, Prometheus, Node.js, Vite, MongoDB, Mongoose, Express, React, Chart.js, TailwindCSS, Vitest

Contact Us

Dominic Kenny – Github | LinkedIn

Connor Donahue – Github | LinkedIn

Wilson Wu – Github | LinkedIn

David Tezza – Github | LinkedIn

License

MIT License

GitHub

View Github

Top comments (0)