What is the difference between a message broker and a message queue?

🌶 Spicy take: “Start with a microservice architecture. Never start with a monolith. There’s a tendency to want to start with a monolithic project because it makes for a great proof of concept. However, scaling it is nearly impossible, and refactoring your monolith to a microservice can take years or at least months.” - Yaniv Ben Hemo

When I asked Yaniv Ben Hemo how people should decide whether to build a monolithic or microservice architecture project, he gave me the above advice. Yaniv’s advice stemmed from his prior experiences building a monolith and struggling to scale it. Also, please note that Yaniv is often building data intensive apps that he feels scale better in a microservice architecture. Eventually, these experiences inspired him to build Memphis, a message broker that enables developers to build microservices easily. You can read more about his passion for building microservices and how to get started here.

Microservices vs. Monoliths

Before I go any further, let’s start by differentiating a monolithic architecture versus a microservices architecture. In a monolithic project, all functions and processes run in a single service. However, as with any project, open source or proprietary, you add more features to the project. This can create bloat and unnecessary complexity. In a microservice architecture, you have multiple, independent components that communicate with each other to run an application. Each microservice is responsible for a single function.

Is Twitter built on a Microservice architecture? Probably!

Here’s an illustration. Although I don’t know the intricacies of Twitter’s design system, I do know that it has multiple features and services. The multi-feature offerings are one of the reasons I enjoy using Twitter. I normally use Twitter to read my timeline, participate in Spaces, interact with tweets, update my profile, subscribe to newsletters, search for past tweets, and view analytics. This doesn’t even scratch the surface of all the features within Twitter. I imagine if Twitter is a monolith, the Twitter engineers are probably stressed! If I were to create a Twitter clone or my own version of Twitter(even to handle the few features I listed), I would build a microservice for each of the following components:

• Analytics Service - This service reports the analytics for each Twitter account
• Search Service - This service allows you to search for tweets and keywords on Twitter.
• Profile Service - This service serves as a profile for each account showing their profile picture, bio, and recent tweet interactions.
• Spaces Service - This service allows you to host, listen, and participate in Twitter Spaces.
• Home Timeline Service - This service allows you to read your timeline.
• Newsletter Service - This service allows you to create and subscribe to newsletters via Revue.
• Tweet Interaction Service - This service allows you to like, retweet, and post tweets.

An image of what Twitter analytics looks like

Each microservice can live in its own repository. This structure allows me to add more features without disrupting the existing features. Also, if a microservice is down, it shouldn’t affect the entire application. If Spaces didn’t work due to a bug, users could still create tweets, read tweets, subscribe to newsletters, search for tweets, and update their profiles.

Microservices use message brokers to communicate

Although microservices work independently, they share data. Let’s use the example of a Twitter Clone microservice. My Tweet Interaction service needs to communicate with the Profile service to know what tweets I liked, retweeted, or responded to, so that it can reflect my likes, retweets, and replies on my profile page. To communicate, microservices use message brokers.

I’ve worked with microservices in the past, but I had many questions about the terminology. The concept of message brokers, message queues, producers, and consumers seemed foreign to me, so I’ll use the rest of this blog post to define these words in layman’s terms.

What is a message queue?

If you took formal Computer Science school, you might’ve heard of the term queue. If not, I’m happy to explain it to you! A queue is similar to a line in real life (like one at a cash register or a line of cars in traffic). Whoever stood in the line first, gets to leave first! This data structure follows a FIFO order, which stands for first in, first out! The first item added to the queue is the first item to leave the queue. Similarly, a message queue contains messages with data about events that follow the FIFO order.

What is a message broker?

When an event occurs, a message with data joins the message queue or “line,” then a message broker, appropriately nicknamed a service bus translates and transports the message to the appropriate service. Message brokers are responsible for managing the transportation and translation of all messages from one microservice to another.

What is a producer?

As the name suggests, producers create and send the data from one microservice to another.

What is a consumer?

Similarly, consumers receive data from a microservice and perform operations based on the data received.

An image of a producer and consumer in Memphis

What is a station?

A station is a term unique to Memphis’ platform. If you’re familiar with Kafka or RabbitMQ, a station is the equivalent of a topic or queue. Stations provide a powerful yet easy-to-use messaging queue for apps.

An image of a station in Memphis' platform

What is a factory?

This is a term that’s unique to the Memphis platform. A factory presents the application/use case that the user requires to build, and, within it, all the stations (queues) that establish the use case.

If you’re interested in better understanding microservices, data, or message brokers, follow Memphis on DEV for more content!

Check out the Memphis repository and give them a star! 🌟