Using the sitecore event queue

In the star trek universe, an almighty species exits, called the Q’s. In the universe of servers, queues also exist. They are used between different servers in order to communicate asynchronously. One server can push messages to a queue while another server (or multiple servers) can read from that queue. Both the writer (the server that pushes messages) as the reader (the server that reads the messages) can work on their own speed because queues keep the unprocessed messages for a while. Queues or also called messaging queues are often depicted as in the figure below, where the left Star Trek Q posts messages into the queue and the right Star Trek Q reads those messages.

Different implementations of this idea exits, e.g. Microsoft service bus, Kafka and the Sitecore event queue. The difference between those solutions is how they are implemented. Because of this, they have different benefits, disadvantages and prices. Sitecore’s solution, called Sitecore event queue, is shipped with a standard Sitecore environment and thus does not require additional costs.

This article will discuss the very basics of the Sitecore event queue followed by a brief summary of how to use it and an explanation of the technical under-the-hood magic just to end with some remarks. Please note that the chapter 2 and 3 are for rather technical minded people.

The basics

The sitecore event queue is implemented as a table (called dbo.EventQueue) and exits in master, web and core databases of a Sitecore solution. When a sitecore instance pushes a message to the queue, a new entry is placed in that table. Because the eventqueue table exists in the three databases, the writing instance should provide to which database should be used. Other (Sitecore) instances poll that table in every database they are connected to, by default every 2 seconds.

How to use it

Usage of the event queue is rather easy, partly because events are just plain C# objects! You can create your own classes with properties to your own needs. These event objects are parsed to json so they can be saved in the event table, so just make sure it is possible to parse them. In order to send an event to the queue, you must first create an event object and then execute the following code:

var database = Sitecore.Configuration.Factory.GetDatabase(“web”);

IEventQueue eventQueue = database.RemoteEvents.EventQueue;

eventQueue.QueueEvent(new CustemEvent(), true, true);

As you can notice, the web database is configured to push the events to. The first Boolean is makes sure that the machine pushing the event may pick up that event from the queue too, this is also called as raising the event locally. The last Boolean is makes sure that other machines may read that event from the queue, also called raising the event globally.

Sitecore instances can listen to the event queue with following code:

var action = new Action<CustomEvent>(HandleCustomEvent);

EventManager.Subscribe(action);

Where HandleCustom event is just a method with a parameter with the type of your custom event class. As you can see, usage is rather simple, just be aware that the instances that need to read your events are connected to the database you are using to push the events.

Technical under-the-hood magic

I’ll briefly explain how the whole queueing system works on a rather technical level. As I discussed before, the event queue is actually a table called dbo.EventQueue, which exists in the three sitecore databases. This table has many different columns including: InstanceType (the type of the event), InstanceName (the name of the machine that raised the event), InstanceData (additional data provided to the event), RaiseGlobally (if it is a globally raised event), RaiseLocally (if the event is a locally raised event) and a Stamp (the timestamp when the event was raised).

In order to know which machine (running Sitecore ofcourse) has already processed which event, a second table is used, called dbo.Properties. This is a very simple table with two columns, one for keys and one for values. This table is used for other things too, but in the case of the event queue, the keys are the machine names and the values are the timestamp of the last processed event.

When instances are polling the event queue they are actually searching the dbo.EventQueue tables in the connected databases for events with a timestamp greater than the timestamp of the last event they processed (which is stored in the dbo.Properties). Events that have the RaiseLocally field is set, are only picked up when they name of the machine is the same of the one that raised the event. On the other hand, events with the RaiseGlobally field are only picked up when the machine name is not equal to the one that raised the event. Events can have both fields set.

Remarks

There are some additional remarks, the first one being that since Sitecore 9.1 CD servers do not have a connection to the core database. So this database cannot be used for the event queue. The second one is that the event queue is cleaned everyday by default (but can be configured). This is in order to keep the event queue table as small as possible.

Sources

https://sitecorebasics.wordpress.com/2011/03/19/basics-of-sitecore-event-queue
https://hishaamn.wordpress.com/2016/02/24/sitecore-event-queues/