DEV Community

loading...
Cover image for Install Kafa to Raspberry Pi 4

Install Kafa to Raspberry Pi 4

nimrodresulta profile image Nimrod Resulta ・2 min read

I wanted to run kafka on my raspberry pi 4 as i wanted to do a POC on an IoT project. Here are the steps on how to install it. I use SSH to login to my Rpi.

1. Install Java:

  • Run the following command to install it:
$ sudo apt update
$ sudo apt install default-jdk
  • Check version to test if the installation is successful. It should show you the version installed as shown below.
$ java --version
openjdk 11.0.8 2020-07-14
OpenJDK Runtime Environment (build 11.0.8+10-post-Raspbian-1deb10u1)
OpenJDK Server VM (build 11.0.8+10-post-Raspbian-1deb10u1, mixed mode)

2. Install Kafka:

$ wget https://apachemirror.sg.wuchna.com/kafka/2.6.0/kafka_2.12-2.6.0.tgz
  • Extract Kafka. Using the tar command. Once extracted, go inside the directory and create /data folder with /kafka and /zookeeper subfolders.
$ tar -xzf kafka_2.12-2.6.0.tgz
$ cd kafka_2.12-2.6.0
~/kafka_2.12-2.6.0 $ mkdir data
~/kafka_2.12-2.6.0/data $ mkdir kafka
~/kafka_2.12-2.6.0/data $ mkdir zookeeper
  • Edit config/zookeeper.properties
$ sudo nano kafka_2.12-2.6.0/config/zookeeper.properties

Find and change dataDir value to /home/pi/kafka_2.12-2.6.0/data/zookeeper.

dataDir=/home/pi/kafka_2.12-2.6.0/data/zookeeper

Press Ctrl+X then Y to save it.

  • Edit config/server.properties
$ sudo nano kafka_2.12-2.6.0/config/server.properties

Find and Change listeners value to PLAINTEXT://{your_ip_address}:9092. Ensure to replace {your_ip_address} with your device ip address.

listeners=PLAINTEXT://{your_ip_address}:9092

Next is to locate and update log.dirs value to /home/pi/kafka_2.12-2.6.0/data/kafka.

log.dirs=/home/pi/kafka_2.12-2.6.0/data/kafka

3. Test Kafka

  • Start Zookeeper
~/kafka_2.12-2.6.0 $ bin/zookeeper-server-start.sh config/zookeeper.properties
  • Start Kafka
~/kafka_2.12-2.6.0 $ bin/kafka-server-start.sh config/server.properties
  • Create a topic
~/kafka_2.12-2.6.0 $ bin/kafka-topics.sh --create --bootstrap-server {your_ip_address}:9092 --replication-factor 1 --partitions 1 --topic TestTopic

It will show you the message below as reponse for topic created.

Created topic TestTopic.

  • Check for the created topic
~/kafka_2.12-2.6.0 $ bin/kafka-topics.sh --list --bootstrap-server {your_ip_address}:9092

This will show you the newly created topic

TestTopic

  • Start Kafka Producer
~/kafka_2.12-2.6.0 $ bin/kafka-console-producer.sh --broker-list {your_ip_address}:9092 --topic TestTopic
  • Start Kafka Consumer
~/kafka_2.12-2.6.0 $ bin/kafka-console-consumer.sh --bootstrap-server {your_ip_address}:9092 --topic NewTopic

Once both Producer and Consumer is up and running start typing on the producer window. Each message should show in the consumer window.

Alt Text

Discussion (0)

pic
Editor guide