We are trying to replace our Prometheus metrics and alarms with the Elastic stack and run into some issues.
For example, with Prometheus, we use the up metric to create an alarm that will tell us if a given service, like etcd, is down. But what will be the equivalent alarm in Elastic?
At the moment we use this condition: WHEN Document count IS ABOVE OR EQUALS 1 and filters not running pods: not kubernetes.pod.status.phase:running and kubernetes.labels.app:etcd
What is considered as best practice for such alarms in Elastic?
Hello @EDzhelyov,
Thanks for reaching out, and welcome to the Elastic community!
The conditions and the filter you shared seem reasonable and should fulfill the purpose.
As you know, the pod status could be Pending,Succeeded,Unknown, or Failed and based on the filter you shared, these statuses will also trigger the alert. So it depends on your desire and how sensitive the rule is; you could be more granular on the pod status. e.g. distinct the list of the healthy status.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.