Azure Service Bus Dead-Letter Queue Monitoring

Azure Service Bus dead-letter Queue

Azure Service Bus queues and topic subscriptions provide a secondary sub-queue, called a dead-letter queue. The purpose of the dead-letter queue is to hold messages that cannot be delivered to any receiver, or messages that could not be processed. So, any message that resides in the dead-letter queue is called a dead-lettered message.

Dead-Letter Queue Reasons

Before investigating the best way to monitor and reprocess Service Bus messages efficiently, let us quickly hop in and understand the reasons behind why an active message get dead-lettered.

Most of the time we could notice that the message fails to reach the receivers due to the following reasons;

1. Dependent service not available
2. Faulty message
3. Process code issue

Dependent service not available

This could be one of the foremost and time after time reasons where the services that reply on message delivery may go down for a short period. For instance, Redis or SQL connection issues may often happen.

Faulty Message

If in case the message doesn’t have a mandatory parameter or some value is incorrect, then the message will end up in the dead-letter queue after the max delivery count is attained.

The failed delivery can also be caused by a few other reasons such as network failures, a deleted queue, a full queue, authentication failure, or a failure to deliver on time.

Here we can drill down the reasons into two ways:

1. System-level dead-lettering
2. Application-level dead-lettering

Reasons for System-level dead-lettering

• Header Size Exceeded – Maximum header size: 64 KB
• Error on processing subscription rule
• Exceeding time to live value
• Exceeding maxDeliveryCount – When DeliveryCount exceeds MaxDeliveryCount, the message is moved to the DLQ, specifying the ‘MaxDeliveryCountExceeded’ reason code. This behaviour cannot be disabled, but you can set MaxDeliveryCount to a very large number. You could specify the Maximum Delivery Count between 1 and 2147483647.
• When Session id property is set to true (the default is false)

Reasons for Application-level dead-lettering

• Messages that cannot be properly processed due to any sort of system issue
• Messages that hold malformed payloads
• Messages that fail authentication when some message-level security scheme is used

Monitoring Service Bus dead-lettered messages made easy

Be it Azure Service Bus Queue or Topic Subscriptions, Serverless360 makes it handy to monitor the dead-letter messages in your testing, pre-production and production environments.

It completely eliminates the complexity or limitation of Azure Monitor which might not fulfill your production environments requirements. For instance,

• Monitor multiple Queues and Topics Subscriptions dead-lettered count in a single report

• Monitor a Queue metrics against another metric, for instance, compare active messages vs dead-lettered messages

• Consolidated monitoring of both the property values and status of Service Bus

I will take you through Serverless360 and showcase how quickly you can configure the alert rule to monitor the dead-lettered messages.

Once you signed up for a free trial, associate the Azure Service Bus in Business Application (Serverless360 feature) – this module lets you access, monitor and manage various Azure resources.

To configure rules, head to the desired resource type in the Manage resources section inside the Business Application. Select the respective queue or topic for which the rules should be configured, and choose the dead-letter count property with warning threshold and error threshold. That’s it.

Azure Service Bus dead letter queue resubmit

Serverless360 makes your life easier when it comes to repairing and resubmitting the dead-letter messages of queue or topic subscription providing out of the box features like

• Advanced search option to find the dead-lettered message
• Intelligent automation to bulk resubmit the dead-lettered messages
• Variety of message receval modes like Peek-lock and Deffered mode
• Back up and purge messages
• Download particular set of messages

Now, look at the step-by-step process to resubmit the dead-lettered messages in Serverless360

• Click the queue whose dead letter messages must be processed, this will lead to a message retrieval dashboard.

• Select dead letter tab and click to get a specified count of messages

• Select those messages to be resubmitted and click resubmit

• Select the destination Queue and proceed to resubmit

• The messages can also be repaired and resubmitted

• The user can also view the properties, content and dead letter reason of a message

Conclusion

I hope this blog gave a better understanding of Azure Service Bus Dead Letter Queue, the reason behind a message getting dead lettered. We also discussed on how Serverless360 can help to process Dead letter messages much smarter way.