DEV Community

loading...
Cover image for  📬 Apache Kafka Commands

📬 Apache Kafka Commands

ricdev2 profile image Ricardo Dev ・2 min read

Currently when we need to connect large information systems with different formats and volumes we will depend on tools that are in charge of being in the middle of our architecture, organizing and distributing all that information operating in high availability, this is where the integration options we have could fit very well Apache Kafka would fit our project very well.

What is Apache Kafka?

Apache Kafka is an open-source distributed event streaming platform used by thousands of companies for high-performance data pipelines, streaming analytics, data integration, and mission-critical applications.
https://kafka.apache.org

Get Apache Kafka

At the time of writing this post it is in version 2.6

https://kafka.apache.org/downloads

Kafka page download

Kafka Architecture

Img

  • Brokers: In charge of containing the topics of the cluster
  • Publishers: They are in charge of putting the messages on the topics
  • Consumers: They extract the messages from the topics to process them
  • Cluster: A set of Kafka nodes
  • Topics: Flow of data that have a same context
  • Partitions: The corresponding divisions of a topic
  • Zookeeper: Orchest all nodes in the cluster

Zookeeper

To use Kafka we need to first start Zookeeper, in the installation folder there is the shell command in the folder bin. Which is:

$ bin/zookeeper-server-start.sh config/zookeeper.properties
Enter fullscreen mode Exit fullscreen mode

Kafka

Now we start Kafka, we need to locate ourselves in the bin directory where the shell is located.

$ bin/kafka-server-start.sh config/server.properties
Enter fullscreen mode Exit fullscreen mode

Creating the topic

To generate a topic we must execute:

$ bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 2 --partitions 3 --topic my-topic 
Enter fullscreen mode Exit fullscreen mode

List all topics

$ bin/kafka-topics.sh --list --zookeeper localhost:2181
Enter fullscreen mode Exit fullscreen mode

Get information about the topic

$ bin/kafka-topics.sh --describe --zookeeper localhost:2181 --topic my-topic 
Enter fullscreen mode Exit fullscreen mode

Delete a topic

$ bin/kafka-topics.sh --delete --zookeeper localhost:2181 --topic my-topic 
Enter fullscreen mode Exit fullscreen mode

Creating a consumer

$ bin/kafka-console-consumer.sh --topic my-topic --from-beginning --bootstrap-server localhost:9092
Enter fullscreen mode Exit fullscreen mode

Creating a producer

$ bin/kafka-console-producer.sh --topic my-topic --bootstrap-server localhost:9092
> Mensaje 1
> Mensaje 2
Enter fullscreen mode Exit fullscreen mode

Discussion (1)

Collapse
andreslopezrm profile image
ALR

Great!!!

Forem Open with the Forem app