Distribution
Kafka consumers get distributed through topic partitions. Each consumer consumes messages from a specific partition at a time.
There are a number of consumers present for each RabbitMQ queue instance. These consumers are known as Competitive consumers as they compete with one another for consuming the message. But, the message can be processed just once.
High Availability
With the help of zookeeper, it manages the state of the Kafka cluster and supports high availability.
Through clustering and high available queues provides high-performance data replication. Thus, RabbitMQ also provides high availability.
Performance
Kafka can process millions of messages in a second with less number of the hardware.
RabbitMQ can also process millions of messages within a second, but it needs more number of the hardware.
Replication
There are replicated brokers available in Kafka, which works when the master broker is down.
Here, queues are not automatically replicated. The configuration is mandatory.
Multi subscriber
Multiple consumer types can subscribe to many messages to Kafka.
In RabbitMQ, although messages are routed to various queues, only one consumer from a queue can process the message.
Message Protocols
Apache Kafka supports primitives such as int8, int16, etc. and binary messages.
RabbitMQ supports any standard queue protocols such as STOMP, AMQP, HTTP, etc.
Message Ordering
In Kafka Message ordering is present inside the partition only. It guarantees that either all fail or pass together.
RabbitMQ maintains the order for flows via a single AMQP channel. In addition, it also reorders the retransmitted packets inside its queue logic that will prevent the consumer from resequencing the buffers.
Message lifetime
Kafka contains a log file that prevents all messages anytime.
Since RabbitMQ is a queue, messages once consumed are removed, and the acknowledgment is received.
Architecture
Highly scalable pub/sub distributed messaging system. It has brokers, topics, partitions, and topics within the Kafka cluster.
A general-purpose pub/sub message broker. Its architecture varies from Kafka as it consists of queues.
Use Cases
Kafka is mainly used for streaming the data.
The web servers mainly use RabbitMQ for immediate response to the requests.
Transactions
Kafka supports those transactions that exhibit a read-process-write pattern performed to/from Kafka topics.
RabbitMQ does not guarantee atomicity even when the transaction indulges only a single queue.
Language
Apache Kafka is written in Scala with JVM.
RabbitMQ is written in Erlang.
Routing Support
Kafka supports complex routing scenarios.
RabbitMQ does not support complex routing scenarios.
Developer Experience
With high growth, Kafka led to a good experience. But, it only supports Java clients.
RabbitMQ carries mature client libraries that support Java, PHP, Python, Ruby, and many more.
Top comments (10)
Have you ever used these tools?
Quite a lot of this article is wrong.
For starters Kafka can be used by anything implementing the protocol it uses ive seen implementations in python, javascript, ruby, and more.
Also the sheer number of vague statements like 'it can process millions of transactions it just needs more numbers of hardware'
What does that even mean?
Please understand what you are writing about before publishing. Articles like this end up encouraging poor architectural choices because managers or devs see flashy articles like this without understanding the information any more than the author did.
@jenkinszevrant Thanks for your valuable comments.
Regarding the mentioned performance related points in the article are referred from some of the reliable online documentations.
educative.io/answers/kafka-vs-rabb...
Feel free to share the correct performance comparison between these two.
No i dont think im going to write your article for you.
If you're going to pose as a subject matter expert you need to actually be an expert on the subject matter.
Also in the future try using the documentation for the actual product instead of some random article, otherwise you're just spreading misinformation. The article you posted is also just outright wrong.
It's not good content if it's wrong and it's just copying data from someone else's article with zero effort to fact check or read the documentation made by the dev team.
Not sure how asking someone to fact check and read the docs before posting information is being a jerk. Articles like this send false information to people who then base decisions on it.
No im not going to write my own article because im not an expert on rabbit mq and never claimed to be. Im just asking the author to put in the effort to be clear, concise, and accurate, especially if those inaccuracies, and vagueness is easily verifyable with minimal effort. If that's too much to ask then the author needs to look for something else to write about.
As i said, dont act like a subject matter expert if you're not actually knowledgeable on the subject. That's not being rude thats having realistic expectations and standards for people who write these articles.
This is may be the worst compression between this two tools.
Kafka is pull based, message log.
RabbitMQ is pub/sub message queue.
While we can use them to implement similar solution, in 90% cases they serve very different needs. And have very different pros and cons.
So to correct compression, two different by purpose andvdesign system you must compare them on same use case. And not just by general properties like message throughput, scaling, ha, message latency, etc.
P.S. Message protocol part of this article is nonsense. I even don't know from point start explain that you try compare apples to pigs.
But briefly you compare Kafka data type(which do not exist by the way, Kafka message value is always binnary) and transport level networks/communication protocol.
Kafka can be used from .Net clients.
Both kafka and rabbitMQ (or AMQP) can be used with .net, .net-core, php, perl, python, javascript, java and many many more.
Please, see dev.to/adderek/kafka-vs-rabbitmq-4073
Not really better eighter. Sorry. The same useless statements about performance and DX.
I tried to replicate every statement from this article... Feel free to suggest changes