AMQP Exchange type comparison, using GO RabbitMQ client.

RabbitMQ is a popular and powerful open-source message broker. I don't want to bother who will read it writing details of the technology behind RabbitMQ, prons and cons or mainly use. I prefer to deepen the features of the Advanced Message Queuing Protocol (AMQP) exploring the differences regarding the Exchange type by using some easy examples.

requirements

You can trust my word, or you can try by your self the examples provided below. In the latter case, this is what you need to launch the examples:

• linux distribution
• docker
• go installed ( my version 1.13.3 )

tl;dr

RabbitMQ works as a mediator, or middleware, that allows software components to communicate. Producer and Consumer represent respectively the one who creates the message and the one who receives the message, don't have to reside on the same network and the interoperability between them allows the use of different technologies. Here, for simplicity, we will work in the same network and everything will be developed in Go.
The Message Broker is responsible for distributing the message from Producer to Consumer, guaranteeing the message received. The atomic unit of the system is the Message, it contains in its body the binary data sent. Message body could be set in different types of formats such as text, XML, JSON and thanks to its binary content stored the data are uniquely interpreted.

Other components involved

We have already faced Producer, Consumer and Message Broker. The communication process described above has been explained too easily. Inside the RabbitMQ, different components allow the procedure to be completed. The message flow starts when the Producer creates the message and sends it to the Broker. The first component the message encounters is the Exchange that routes it into zero or more Message Queue, according to the Exchange type and matching between the routing key of the message and the binding key of the Bindings. Binding is something like a link or a relationship between Queue and Exchange. Message Queue stores messages in memory or disk and delivers messages to the Consumer.
So, the Exchange is the element responsible for delivers of the messages by applying a routing algorithm. The core of this post is to explain how the Exchange behavior changes comparing these types: fanout, direct and topic.

core of the post

Let's try to apply RabbitMQ to an application that can be easily explained, or at least that's the easiest solution I imagined.
The environment is a home automation system able to control its peripherals (like a bulb light) by receiving messages through the RabbitMQ broker. So the Broker could be an interface between some apps (Producer) and the actuators of the house (Consumer).

project tree

Here the link where to find the repository.

https://github.com/enbis/learning-rabbitmq

The project tree is shown below. The folders referring to the cases dealt in this post are the four exposed. The other folders are sided tests.

.
├── acknowledgment
├── direct-exchange
│   ├── config.yml
│   ├── consumer
│   │   ├── consumer.go
│   │   └── consumer_test.go
│   ├── makefile
│   ├── producer
│   │   ├── producer.go
│   │   └── producer_test.go
│   └── README.md
├── durability
├── fair-dispatch
├── fanout-exchange
│   ├── config.yml
│   ├── consumer
│   │   ├── consumer.go
│   │   └── consumer_test.go
│   ├── makefile
│   ├── producer
│   │   ├── producer.go
│   │   └── producer_test.go
│   └── README.md
├── global
│   ├── config
│   │   └── config.yml
│   ├── connection
│   │   └── handler.go
│   ├── models
│   │   └── models.go
│   └── utils
│       └── utils.go
├── README.md
├── round-robin-queuing
├── start-messaging
└── topic-exchange
    ├── config.yml
    ├── consumer
    │   ├── consumer.go
    │   └── consumer_test.go
    ├── makefile
    ├── producer
    │   ├── producer.go
    │   └── producer_test.go
    └── README.md

All the examples refer to the global folder for the functions of common use like the connection handler or the utils. Within each folder, you will find the makefile, config file, and the README with some instructions to execute the example.

docker

Be careful pulling the correct version of the docker image, to be able to inspect the useful RabbitMQ UI on localhost:15672.

docker run --rm --name myrabbitmq -p 5672:5672 -p 15672:15672 rabbitmq:3-management

fanout-exchange

Fanout Exchange type is only capable of broadcasting, it ignores the Binding routing-key and sends messages to each Queue it knows. So, the messages sent by the Producer arrive at the Exchange, which forwards them to the Message Queue unconditionally.

Applying it to the home automation example, we can imagine having two Queues operating two distinct rooms. The Consumer can be imagined as a light actuator located inside each room. As soon as the Producer sends the commands to the Exchange, the messages will be forwarded to both Queue with the result that the action to turn on or off the light will be executed simultaneously in both rooms.

To run the test

From within the fanout-exchange folder

• Terminal 0 -> make test/consumer/room0 -> It runs the Consumer#0, as room0.
• Terminal 1 -> make test/consumer/room1 -> It runs the Consumer#1, as room1.
• Terminal 2 -> make test/producer -> It runs the Producer, that sends messages to switch on / off the light bulb.

It will produce the following results

Terminal 2 -> The Producer

0: Light bulb On
1: Light bulb Off
2: Light bulb On
3: Light bulb Off
4: Light bulb On

Terminal 0 -> The Consumer#0 as a Room0

Waiting messages
message received: Light bulb On
message received: Light bulb Off
message received: Light bulb On
message received: Light bulb Off
message received: Light bulb On

Terminal 1 -> The Consumer#1 as a Room1

Waiting messages
message received: Light bulb On
message received: Light bulb Off
message received: Light bulb On
message received: Light bulb Off
message received: Light bulb On

direct-exchange

Direct Exchange handles a simple routing algorithm: a message goes to the Queues whose binding key exactly matches the routing key of the message. That type of Exchange is able to forward data selectively, on the other side it can't do routing based on multiple criteria. So, the Exchange receives the message with routing key R and sends it to Queue bound to the Exchange with binding key B only if R and B are equals.

Applying it to the home automation example, again we can imagine having two Queues operating two distinct rooms. The Queue related to the first room is bound to the Exchange using routing key all_rooms and first_room; the Queue related to the second room is bound to the Exchange using routing key all_rooms and second_room. So, messages with routing key all_rooms are routed to both Queues, rather messages with routing key first_room or second_room are routed respectively to first Queue or the second Queue. In our imaginary home automation system, we can think to send the command to switch on the light with routing key all_rooms, and switch off them selectively using the other two routing keys.

To run the test

From within the direct-exchange folder

• Terminal 0 -> make test/consumer/room0 -> It runs the Consumer#0, binding with all_rooms and first_room routing key.
• Terminal 1 -> make test/consumer/room1 -> It runs the Consumer#1, binding with all_rooms and second_room routing key
• Terminal 2 -> make test/producer -> It runs the Producer. It sends 3 messages: Bulb On, Bulb Off and Bulb Off. Each message is destined to the proper Queue, depending on the binding key.

It will produce the following results

Terminal 2 -> The Producer

0: Light bulb On to routingKey all_rooms
1: Light bulb Off to routingKey first_room
2: Light bulb Off to routingKey second_room

Terminal 0 -> The Consumer#0 as a Room0

Binding to  all_rooms
Binding to  first_room
Waiting messages

message received: Light bulb On
message received: Light bulb Off

Terminal 1 -> The Consumer#1 as a Room1

Binding to all_rooms
Binding to second_room
Waiting messages

message received: Light bulb On
message received: Light bulb Off

topic-exchange

Topic Exchange handles a complex routing algorithm, leveraging on messages with routing key composed by a list of words delimited by dots. The logic behind the topic Exchange is not so different from the direct Exchange, a message containing a rounding key will be delivered to all Queues that are bound to the Exchange with a matching binding key pattern. The difference with respect to the previous case lies in the fact that in this case there are special characters that allow a greater spectrum of combinations.

• * star substitutes exactly one word of the key pattern
• # hash substitutes for zero or more words of the key pattern

Applying it to the home automation example, we have a broader case study to work on. Firstly we need to specify the categories used to define the pattern. Thinking about a list of three words, we can identify the categories as follow: <building>.<room>.<target>. Now we can imagine the Queues being addressed to a group of different actuators: the first one will act on the lights, the second on everything related to the garage area (lights included). So, back to the binding patterns, the first Queue will handle this pattern #.lights to specify that is interested in everything related to the implementation of the lights, rather the second Queue receives all commands related to the garage: *.garage.*. In that situation, all the messages that have routing key with lights as last routing key parameters will be forwarded by the first Queue; while all the messages containing garage as a second routing key parameter are intended to the second Queue. In some cases, the messages will be forwarded to both the Queues.

This is the list messages tested

• routing key house.room1.light, with body On -> direct to lights only
• routing key house.garage.light, with body On -> direct to both Consumers
• routing key house.garage.door, with body Open -> direct to garage only
• routing key house.backyard.irrigation, with body On -> neither of two consumers receives the message
• routing key house.garage.light.desktopLamp, with body On -> neither of two consumers receives the message, the pattern format is not recognizable.

The Consumer uses simple strings.Split() function to read the routing key and understand the action contained in the body of the message.

go func() {
    for msg := range msgs {
        s := strings.Split(string(msg.RoutingKey), ".")
        fmt.Println("Where: " + s[1])
        fmt.Println("What: " + s[2])
        fmt.Println(string(msg.Body))
        fmt.Println("----------")
    }

}()

To run the test

From within the topic-exchange folder

• Terminal 0 -> make test/consumer/lights -> It runs the Consumer#0, that handles this binding pattern: #.light.
• Terminal 1 -> make test/consumer/garage -> It runs the Consumer#1, that handles this binding pattern: *.garage.*
• Terminal 2 -> make test/producer -> It runs the Producer, sending 5 messages with different routing key.

It will produce the following results

Terminal 2 -> The Producer

0: value 0 Action: On to routingKey house.room1.light
1: value 1 Action: On to routingKey house.garage.light
2: value 2 Action: Open to routingKey house.garage.door
3: value 3 Action: On to routingKey house.backyard.irrigation
4: value 4 Action: On to routingKey house.garage.light.desktopLamp

Terminal 0 -> The Consumer#0

qbinding  #.light
Waiting messages

Where: room1
What: light
0 Action: On
----------
Where: garage
What: light
1 Action: On
----------

Terminal 1 -> The Consumer#1

qbinding  *.garage.*
Waiting messages

Where: garage
What: light
1 Action: On
----------
Where: garage
What: door
2 Action: Open
----------

conclusion

After all this, my personal advice is to keep using your hands to turn the house lights on and off 🙌💡