DEV Community

Mirco
Mirco

Posted on • Originally published at dashdashverbose.Medium

Monitoring setup with docker-compose - Part 3: Alertmanager

In short: What is Alertmanager?

The Alertmanager sends alerts to various channels like Slack or E-Mail.
Recall from part one that Prometheus creates an alert if something violates a rule.

You can use the Alertmanager to silence and group alerts as well.

Configuration

You can get all source code from GitHub. Check out the tag part-2-grafana if you want to follow along.

docker-compose.yml

First of all, add the Alertmanager and a volume to docker-compose.yml:


  alertmanager:
    image: prom/alertmanager:v0.23.0
    restart: unless-stopped
    ports:
      - "9093:9093"
    volumes:
      - "./alertmanager:/config"
      - alertmanager-data:/data
    command: --config.file=/config/alertmanager.yml --log.level=debug


volumes:

  alertmanager-data:

Enter fullscreen mode Exit fullscreen mode

Alertmanager will persist silence configurations to the volume.

alertmanager/alertmanager.yml

This configuration contains information about which channels to send to. For simplicity, we use E-Mail. Refer to the Alertmanager docs to learn about other channels.

Create a folder alertmanager and add a file alertmanager.yml to it:

route:
  receiver: 'mail'
  repeat_interval: 4h
  group_by: [ alertname ]


receivers:
  - name: 'mail'
    email_configs:
      - smarthost: 'smtp.gmail.com:465'
        auth_username: 'your_mail@gmail.com'
        auth_password: ""
        from: 'your_mail@gmail.com'
        to: 'some_mail@gmail.com'
Enter fullscreen mode Exit fullscreen mode

The route section configures which alerts will be sent. In our case, we sent all alerts. You could add more routes and filter, for example, on alert tags (see the example in the docs.).

receivers configures our target channels. Note how route refers to the receiver mail on line two.

Connect to prometheus

Finally, we need to tell Prometheus about the Alertmanager.

Open prometheus/prometheus.yml and add the following:

alerting:
  alertmanagers:
    - scheme: http
      static_configs:
        - targets: [ 'alertmanager:9093' ]

Enter fullscreen mode Exit fullscreen mode

Testing

Run docker-compose up. Open http://localhost:9093 in your browser to see the Alertmanager UI.

Alertmanager after start

After a couple of minutes, the test alert fires. You can check this in your Prometheus instance.

Prometheus with active alert

Now, you can see the alert in the http://localhost:9093 as well:

Alertmanager with alert

Check your inbox for the notification:
Alert mail

Final note

This series shows how to set up a basic Prometheus stack. Keep in mind that this is not production-ready! But it is enough to get your hands dirty.

Discussion (0)