Heartbeat Alerting Question

Elastic Stack Beats
,
1

Hello,

I am trying to find out a way to do alerting on downed hosts with watcher though I am running into an issue trying to figure out how to handle throttling the alerts. Once a given host is down I only want it to send one email alert then throttle for 24 hours. But I still want it to alert if a new host goes down meaning I can't really use the throttle feature.

One solution could be to make a watcher for each host but I don't know what the scalability/performance impact of that would be.

Has anyone else ran into a similar situation with alerting using watcher?

2

@Kyle123 It used to be true that creating a watch per host didn't scale. However, that is no longer the case, and should work just fine!

3

Since Elasticsearch 6.0, watches are running distributed across the cluster, so you can run far more watches at parallel.

Can you indicate about how many hosts you want to monitor?

4

We are looking to monitor around 250 hosts.

5

you can easily do that on a per host base.