Server Availability Monitor via Metricbeat

[devans (Dave Evans)]

I want to monitor server availability by alerting if a server hasn't sent metricbeat stats within a given interval. So, for example, if ServerA's max timestamp in the metricbeat index is >5min send a warning, and >10 send critical alert.

I also want to display these results in a Kibana visualization(something like the prelert swimlane).

Can this be accomplished with a single watch, or is a separate watch needed for each server?
If a watch action is being throttled, can a change still trigger an alert? For example, if ServerA goes down and triggers an action, then ServerB goes down 10 minutes later, can the action be triggered again?

If multiple watches are needed, can an action be used to create a watch? For example, if a new server appears in the metricbeat index have an action create a new monitoring watch for it.

[Christian_Dahlqvist]

There is an example watch in the examples repository on GitHub that seems similar to what you are looking for and can be used as a starting point.

[devans (Dave Evans)]

Thank you. That was a good help to get me started. I didn't know about the examples repository.

For the index_action, is it possible to index new data to a different index, instead of the payload?

So, if I would want to index

{
"host": "ServerA"
"server_health": 0
}

to a different index.