Delayed requeuing with RabbitMQ

The problem

Your consumer can not handle the received message and you need to reject it. If you reject the message with requeue being true, it will be instantly redelivered to your consumer, resulting in very high workload, as your consumer will reject that message. Things have come full circle 😋

Basicaly you want to reject the message and requeue it, but with a delay.

The solution

Well, you could not reject the message, but you could deliver it to another queue from your consumer and write a cronjob moving this messages back to the working queue. But there should be a better solution, and it should be possible to implement without any changes to your consumer or your message broker.

The real solution

Dead-Letter-Exchange (DLX) to the rescue!

For every queue you can specify a DLX, this is where messages that got rejected or deleted will be delivered to. In combination with a TTL you can automate delivery between queues with a delay.

To make this work, setup your exchanges and queues like so:

• create an exchange named stuff and direct bind to the queue stuff
• create an exchange named stuff-retry and direct bind to the queue stuff-retry
• create a work queue named stuff with the x-dead-letter-exchange header set to stuff-retry
• create a work queue named stuff-retry with the x-dead-letter-exchange header set to stuff and the x-message-ttl header to 300000

That’s it. If your consumer listening on stuff now rejects a message without requeue, it will be delivered to the stuff-retry-exchange which passes the message on to the stuff-retry-queue. After five minutes in that queue, the messages dies and will be redelivered to the stuff-exchange, which passes the message on to the stuff-queue so your consumer can process that message.