DEV Community

Cover image for Install Kafa to Raspberry Pi 4
Nimrod Resulta
Nimrod Resulta

Posted on • Updated on

Install Kafa to Raspberry Pi 4

I wanted to run kafka on my raspberry pi 4 as i wanted to do a POC on an IoT project. Here are the steps on how to install it. I use SSH to login to my Rpi.

1. Install Java:

  • Run the following command to install it: ```

$ sudo apt update
$ sudo apt install default-jdk

+ Check version to test if the installation is successful. It should show you the version installed as shown below.
Enter fullscreen mode Exit fullscreen mode

$ java --version
openjdk 11.0.8 2020-07-14
OpenJDK Runtime Environment (build 11.0.8+10-post-Raspbian-1deb10u1)
OpenJDK Server VM (build 11.0.8+10-post-Raspbian-1deb10u1, mixed mode)

##2. Install Kafka:
+ Download kafka. Goto to [https://apachemirror.sg.wuchna.com/kafka/](https://apachemirror.sg.wuchna.com/kafka/) site and select the latest version. At the time that i was writing this post the latest is 2.6.0.
Enter fullscreen mode Exit fullscreen mode

$ wget https://apachemirror.sg.wuchna.com/kafka/2.6.0/kafka_2.12-2.6.0.tgz

+ Extract Kafka. Using the tar command. Once extracted, go inside the directory and create /data folder with /kafka and /zookeeper subfolders.
Enter fullscreen mode Exit fullscreen mode

$ tar -xzf kafka_2.12-2.6.0.tgz
$ cd kafka_2.12-2.6.0
~/kafka_2.12-2.6.0 $ mkdir data
~/kafka_2.12-2.6.0/data $ mkdir kafka
~/kafka_2.12-2.6.0/data $ mkdir zookeeper

+ Edit config/zookeeper.properties
Enter fullscreen mode Exit fullscreen mode

$ sudo nano kafka_2.12-2.6.0/config/zookeeper.properties

  Find and change **dataDir** value to **/home/pi/kafka_2.12-2.6.0/data/zookeeper**. 

> dataDir=/home/pi/kafka_2.12-2.6.0/data/zookeeper

  Press **Ctrl+X** then **Y** to save it.

+ Edit config/server.properties
Enter fullscreen mode Exit fullscreen mode

$ sudo nano kafka_2.12-2.6.0/config/server.properties

  Find and Change **listeners** value to **PLAINTEXT://{your_ip_address}:9092**. Ensure to replace *{your_ip_address}* with your device ip address.

>listeners=PLAINTEXT://{your_ip_address}:9092

  Next is to locate and update **log.dirs** value to **/home/pi/kafka_2.12-2.6.0/data/kafka**.

> log.dirs=/home/pi/kafka_2.12-2.6.0/data/kafka

##3. Test Kafka
* Start Zookeeper
Enter fullscreen mode Exit fullscreen mode

~/kafka_2.12-2.6.0 $ bin/zookeeper-server-start.sh config/zookeeper.properties

* Start Kafka
Enter fullscreen mode Exit fullscreen mode

~/kafka_2.12-2.6.0 $ bin/kafka-server-start.sh config/server.properties

* Create a topic
Enter fullscreen mode Exit fullscreen mode

~/kafka_2.12-2.6.0 $ bin/kafka-topics.sh --create --bootstrap-server {your_ip_address}:9092 --replication-factor 1 --partitions 1 --topic TestTopic

It will show you the message below as reponse for topic created.
> Created topic TestTopic.

* Check for the created topic
Enter fullscreen mode Exit fullscreen mode

~/kafka_2.12-2.6.0 $ bin/kafka-topics.sh --list --bootstrap-server {your_ip_address}:9092

This will show you the newly created topic
> TestTopic

* Start Kafka Producer
Enter fullscreen mode Exit fullscreen mode

~/kafka_2.12-2.6.0 $ bin/kafka-console-producer.sh --broker-list {your_ip_address}:9092 --topic TestTopic

* Start Kafka Consumer
Enter fullscreen mode Exit fullscreen mode

~/kafka_2.12-2.6.0 $ bin/kafka-console-consumer.sh --bootstrap-server {your_ip_address}:9092 --topic TestTopic


Once both Producer and Consumer is up and running start typing on the producer window. Each message should show in the consumer window.

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/8djq9pw6h213gbq31j19.png)
Enter fullscreen mode Exit fullscreen mode

Top comments (2)

Collapse
 
moonraker727 profile image
Alex

Hello from 2021. I created an account here just to say thanks... but also Start Kafka Consumer should listen to the same topic as the producer, you've got TestTopic and NewTopic. Thanks again, this really helped.

Collapse
 
nimrodresulta profile image
Nimrod Resulta • Edited

Hey, thanks man, appreciate it. I have updated my post.

Some comments may only be visible to logged-in visitors. Sign in to view all comments.