Watcher help send mutple alerts

Elastic Stack Elasticsearch
1

Hello

First time writing my own watcher and need some help please, I want it to alert if the system load on an host is grate then a set number for longer then 5 minutes. (for testing on my environment, i set it to an low number "0.05"), but when i action it i get 5 hosts matching the result

Is their any way to set the action to create an alert (say over slack) to alert all hosts with their host names?

My watch i crated so fair

{
  "trigger": {
"schedule" : {
  "interval" : "10s"
}
  },
  "input":   {
"search" :{
  "request": {
    "search_type": "query_then_fetch",
    "indices": "metricbeat-*",
    "body":{
      "size": 0,
      "query": {
        "bool":{
          "filter": [
            {
              "range":{
                "@timestamp":{
                  "from": "now-5m",
                  "to": "now"
                }
              }
            }
          ]
        }
      },
      "aggs": {
        "hostname":{
          "terms" :{
            "field": "host.hostname"
          }
        }
      } 
    }
  }
  
}
  },
  "condition": {
"compare": {
  "ctx.payload.aggregations.hostname.buckets.*.system.load.1": {"gt": 0.05}
}
  },
  "actions": {}
}
2

first, your compare condition will not work. You should use a script condition and have a script doing that check.

And yes, you can use slack the inform about alerts, see https://www.elastic.co/guide/en/elastic-stack-overview/6.6/actions-slack.html

3

Thanks i look at the script part latter.

I know i can use slack, i am more thinking if watcher comes back with 3 hosts, i would like 3 slack alerts.

Sorry if i am not making sence

4

sorry, that part I indeed misunderstood. Right now you can only have one alert aka one slack message. If you really need more, you could send that data over to logstash and then create several alerts on the logstash side using the http output.

5

Thanks For that, I am trying to monitor system load, for each server in my environment.
If any goes over the set limit then i would receive an watch alert.

I am seeing when running this watch, that i get an list of 4 servers matching the limit and would like 4 alerts.

Can i ask

If i can write one watcher alert to monitor the servers and alert when any goes over the limit
or
Would i need to write an watcher alert for each host

6

If you need to send those four alerts for example to different slack channels, you would need to have four different alerts.

If a single message consisting of four different attachments is sufficient, then a single alert is the way to go.

--Alex

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.