Monitoring and Logging Docker Events

There are many Docker logging platforms available with agents that can be installed to track metrics and monitor the performance and availability of your clusters.

But sometimes it’s just not worth the costs of such services depending on the scale of your operations. A more feasible option could be hosting your own monitoring and logging solution using Prometheus, Grafana, Alertmanager, cAdvisor, and NodeExporter.

If you still feel like it’s an overkill, you may want to settle down for a quick-and-dirty solution to monitor Docker events and send the logs to your destination of choice.

Docker has an API that allows listening to its events and comes with an array of options such as filtering based on conditions like event type and formatting the output to your liking.

With a little bit of bash scripting, we can put together a simple service that listens for particular event statuses and triggers a cURL call to alert you.

For this example, I’m filtering all events leading to containers shutting down either normally or due to an issue such as running out of memory.

Create the docker-monitor.sh script:

#!/bin/bash

# Monitor all Docker events
docker events --filter 'event=die' --filter 'event=exec_die' --filter 'event=kill' --filter 'event=stop'   --format '{{.ID}}' | while read event

do
    container_id=$event
    # Get the image name of the started container
    image=`docker inspect --format='{{.Name}}' $container_id`
    # Send a cURL call to alert you that something's not right!
    curl https://mywebsite.com/log
done

Create a service at /etc/systemd/system/docker-monitor.service to automatically start the script:

[Unit]
Description=Docker Monitor Service

[Service]
Type=forking
ExecStart=/path/to/docker-monitor.sh

[Install]
WantedBy=multi-user.target

Reload systemd:

systemctl daemon-reload

Start your service:

systemctl start docker-monitor

Enable the service for automatic start on reboot:

systemctl enable docker-monitor

This post was originally published on my blog where I write all about tech.