Apache Kafka is a fast and scalable messaging queue, capable of handling heavy loads in context of read and write, i.e. IO-related, stuff. You can find more about Kafka on http://kafka.apache.org/. Apache Kafka requires a running ZooKeeper instance, which is used for reliable distributed coordination. Please find more about ZooKeeper on https://zookeeper.apache.org/.
Downloading the Required Files
- Download Server JRE according to your OS and CPU architecture from http://www.oracle.com/technetwork/java/javase/downloads/jre8-downloads-2133155.html
- Download and install 7-zip from http://www.7-zip.org/download.html
- Download and extract Kafka using 7-zip from http://kafka.apache.org/downloads.html
For this tutorial, we are assuming that kafka_2.11-2.4.0 are unzipped in the C: drive, but you can unzip them in any location.
Starting of Zookeeper and Apache Kafka
- I will skip Java installation part as people are familiar with it.
- We can start the server with default configuration and here is the following command. zookeeper-server-start.bat ../../config/zookeeper.properties
- Now we will start Apache Kafka Server with following command. kafka-server-start.bat ../../config/server.properties
- Now we will create a topic with the name “test” and a replication factor of 1, as we have only one Kafka server running. If you have a cluster with more than one Kafka server running, you can increase the replication-factor accordingly, which will increase the data availability and act like a fault-tolerant system. kafka-topics.bat --create --zookeeper localhost:2181 --replication-factor 1 - -partitions 1 --topic sample
- Now we will create Producer and Consumer to test server. kafka-console-producer.bat --broker-list localhost:9092 --topic sample
- Now we will start Consumer by typing the following command. kafka-console-consumer.bat --zookeeper localhost:2181 --topic sample Now we can send Messages from Producer and Consumer consumes it.
- Here are the useful commands that would be helpful.
- To list topics kafka-topics.bat --list --zookeeper localhost:2181.
- To describe topic kafka-topics.bat --describe --zookeeper localhost:2181 - -topic [Topic Name].
- Read messages from the beginning kafka-console-consumer.bat --bootstrap- server localhost:9092 --topic [Topic Name] --from-beginning.
- To delete a topic kafka-run-class.bat kafka.admin.TopicCommand --delete -- topic [topic_to_delete] --zookeeper localhost:2181.
Let me know if you are facing any issues in discussion session.
Top comments (0)