I wanted to run kafka on my raspberry pi 4 as i wanted to do a POC on an IoT project. Here are the steps on how to install it. I use SSH to login to my Rpi.
1. Install Java:
- Run the following command to install it:
$ sudo apt update
$ sudo apt install default-jdk
- Check version to test if the installation is successful. It should show you the version installed as shown below.
$ java --version
openjdk 11.0.8 2020-07-14
OpenJDK Runtime Environment (build 11.0.8+10-post-Raspbian-1deb10u1)
OpenJDK Server VM (build 11.0.8+10-post-Raspbian-1deb10u1, mixed mode)
2. Install Kafka:
- Download kafka. Goto to https://apachemirror.sg.wuchna.com/kafka/ site and select the latest version. At the time that i was writing this post the latest is 2.6.0.
$ wget https://apachemirror.sg.wuchna.com/kafka/2.6.0/kafka_2.12-2.6.0.tgz
- Extract Kafka. Using the tar command. Once extracted, go inside the directory and create /data folder with /kafka and /zookeeper subfolders.
$ tar -xzf kafka_2.12-2.6.0.tgz
$ cd kafka_2.12-2.6.0
~/kafka_2.12-2.6.0 $ mkdir data
~/kafka_2.12-2.6.0/data $ mkdir kafka
~/kafka_2.12-2.6.0/data $ mkdir zookeeper
- Edit config/zookeeper.properties
$ sudo nano kafka_2.12-2.6.0/config/zookeeper.properties
Find and change dataDir value to /home/pi/kafka_2.12-2.6.0/data/zookeeper.
dataDir=/home/pi/kafka_2.12-2.6.0/data/zookeeper
Press Ctrl+X then Y to save it.
- Edit config/server.properties
$ sudo nano kafka_2.12-2.6.0/config/server.properties
Find and Change listeners value to PLAINTEXT://{your_ip_address}:9092. Ensure to replace {your_ip_address} with your device ip address.
listeners=PLAINTEXT://{your_ip_address}:9092
Next is to locate and update log.dirs value to /home/pi/kafka_2.12-2.6.0/data/kafka.
log.dirs=/home/pi/kafka_2.12-2.6.0/data/kafka
3. Test Kafka
- Start Zookeeper
~/kafka_2.12-2.6.0 $ bin/zookeeper-server-start.sh config/zookeeper.properties
- Start Kafka
~/kafka_2.12-2.6.0 $ bin/kafka-server-start.sh config/server.properties
- Create a topic
~/kafka_2.12-2.6.0 $ bin/kafka-topics.sh --create --bootstrap-server {your_ip_address}:9092 --replication-factor 1 --partitions 1 --topic TestTopic
It will show you the message below as reponse for topic created.
Created topic TestTopic.
- Check for the created topic
~/kafka_2.12-2.6.0 $ bin/kafka-topics.sh --list --bootstrap-server {your_ip_address}:9092
This will show you the newly created topic
TestTopic
- Start Kafka Producer
~/kafka_2.12-2.6.0 $ bin/kafka-console-producer.sh --broker-list {your_ip_address}:9092 --topic TestTopic
- Start Kafka Consumer
~/kafka_2.12-2.6.0 $ bin/kafka-console-consumer.sh --bootstrap-server {your_ip_address}:9092 --topic NewTopic
Once both Producer and Consumer is up and running start typing on the producer window. Each message should show in the consumer window.
Discussion (0)