logo

Alerting in the Prometheus Ecosystem: The Past, Present and Future

2022-05-19

Authors:   Josue Abreu


Summary

The speaker discusses the benefits of using the Alert Manager within Grafana and the importance of open source software in the DevOps community.
  • The Alert Manager complements Grafana's data model and allows for grouping, routing, silencing, and time of day restrictions for alerts.
  • Open source software is important for collaboration and giving back to the community.
  • Improvements to the Alert Manager are being made through feedback and discussion within the community.
  • Future plans include improving visibility in the notification pipeline and creating a historical view of alerts.
  • The speaker emphasizes the value of the Alert Manager and encourages involvement in the DevOps community.
The speaker describes the challenge of lack of visibility in the notification pipeline and the need for information about why alerts are not being delivered. They propose a solution of accessing logs and creating a UI that displays information about the state of alerts and notifications. This would allow for easier debugging and troubleshooting of issues. The speaker also shares their ambition of creating a historical view of alerts, which would provide insight into what was received and notified in the last 48 hours. They acknowledge that this is a complex problem, but are committed to finding a solution through collaboration within the community.

Abstract

One of the most important functionalities of Prometheus is being able to alert based on your metrics. The Prometheus Alertmanager is a critical piece of cloud native observability, and in this talk, Josue wants to share a bit more of its past, present and future. About a year ago, he set out on a path to improve scaling in the Cortex Alertmanager component, then he decided to include the Alertmanager within Grafana to continue fostering open source collaboration. For the future, his plan is to take all the good parts of what he learned on this journey back to the Prometheus Alertmanager thus going full cycle. He’ll cover: The Prometheus Alertmanager and the benefits of its modular architecture (past) The benefits of the new architecture of the Cortex Alertmanager: Like Cortex but for Alerts (past) Inclusion of the Prometheus Alertmanager within Grafana (present) The future of the Prometheus Alertmanager (future).Click here to view captioning/translation in the MeetingPlay platform!

Materials: