How to create alert when some component stopped to send data

Elastic Stack Elasticsearch
1

I'm thinking about using X-Pack Watcher for alerting when some of our components stopped to send data to ELK.
I.e. there is no new data from certain source recently.

  • There is quick&dirty solution by just create individual watcher per data source (i.e. "host", "type", ...). Seems really ugly.
  • Combine different sources in one watcher using array. A little better, but still static list.
  • "We saw messages from these sources in the past - let's check if we see them recently" - I'm not sure what is the best way to implement this logic.
  • ML ? How?

What do you think?
I'm curious if someone already implemented similar thing.
TIA,
Vitaly

2

Hi,
If you have data that should get into the cluster regulary you may build a watcher that check how old is the timestamp of the last inserted entry. If > than x then fire alarm.

pts0

1 Like
3

ML can easily do this with a job that would use the low_count detector. If the data source is a single index, then a Single Metric job will do the trick.

If there are multiple "types" in the index, then the Multi-metric Job is the right choice, splitting on the type field (or whatever you want to split on)

Also, in v5.5, ML makes it easy to create Watches from the Single-Metric and Multi-metric jobs. See this blog: https://www.elastic.co/blog/alerting-on-machine-learning-jobs-in-elasticsearch-v55

2 Likes
4

Many thanks, this article is really good.
I started to play with ML. I'll use Multi Metric job, split by type, as you suggested.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.