How to keep your Amazon MQ queues clean

Amazon MQ queues might fill up if you use them in your tests but don’t take care of cleaning them up. Let’s explore together a way of addressing this issue.

I was hoping to avoid writing dedicated code to just consume all the messages enqueued during tests so I started looking around for some tool I could integrate in my Continuous Integration pipeline.

I found amazonmq-cli. I have to say, it’s not the most straightforward tool when used in a CI pipeline. When it comes to command line options it’s not very flexible and it favours the use of command files to read from. It also only allows for file-based configuration.

Nevertheless, I downloaded and configured it.

Configuring amazonmq-cli in your CI pipeline

Amazon MQ does not support JMX but if your broker does, you could give activemq-cli a try.

For the above reason, this tool needs Web Console access, so make sure that’s configured.

Since I’m running this command as part of a clean-up job in my CI pipeline, I have to automate its configuration.

We need to produce two files:

• the configuration file to be able to access the broker (and the web console)
• and the list of commands to execute

Once you download and extract the tool you’ll see its directory structure is as follows.

➜ tree -L 1
.
├── bin
├── conf
├── lib
└── output

4 directories, 0 files

There’s not much opportunity for customization, so we will have to produce the relevant configuration file and put it in the conf folder (where you can find a sample one).

First thing, then, is to create the broker configuration.

echo -e "broker {\n aws-broker {\n web-console = \"$ACTIVEMQ\_WEB\_CONSOLE\_URL\"\n amqurl = \"$ACTIVEMQ\_BROKER\_URL\"\n username = \"$ACTIVE\_MQ\_USER\"\n password = \"$ACTIVE\_MQ\_PASSWORD\"\n prompt-color = \"light-blue\"\n }\n}\n\nweb-console {\n pause = 100\n timeout = 300000\n}" > /home/ciuser/amazonmq-cli/conf/amazonmq-cli.config

As you can see, I’ve also included a web-console part that is expected by the tool when performing certain commands like the purge-all-queue one that I needed in this case.

Now, let’s configure the list of commands to run.

In my case all the queues generated in the CI environment during the test share the same prefix. This makes it easier for me to purge them all or delete them later.

echo -e "connect --broker aws-broker\npurge-all-queues --force --filter $MY\_SPECIAL\_PREFIX\n" > purge-commands.txt

Drain them!

Finally, we can perform the command.

/home/ciuser/amazonmq-cli/bin/amazonmq-cli --cmdfile $(pwd)/purge-commands.txt

That’s it! Let me know your experiences with cleaning up queues on Amazon MQ!

The post How to keep your Amazon MQ queues clean appeared first on Alessandro Diaferia.