Creating a watcher with more conditions - how to

Elastic Stack Elasticsearch
1

Hey.

Imagine you have the following data set:

{
  "_index": "myindex",
  "_type": "_doc",
  "_id": "0f3e7c49b480a4d7ea2352a424ae8550",
  "_version": 1,
  "_score": null,
  "_source": {
    "Measurement.Warning": 0.2,
    "Payload.Created": "2020-09-24T11:14:48Z",
    "Location.Station": 20,
    "Measurement.Value": 0.5,
     }
}

Lets say, the "Measurement.Value" changes. Image you got more than one "Location.Station". But all data goes to the same elastic index.

I like to create a single watcher (or if not possible a watcher each station) ... to do the following:

each time a "Measurement.Value" bigger than "Measurement.Warning" is written to the index, I want to send a mail to a specified group of people. I want to include a specific text like:

'station 20 exceeded the maximum of 0.2 with the current value of 0.5'
and if possible in the caption of the mail: 'warning for station 20'

currently I'm trying like this, but I didn't find a way to split by station and to ad specific text.

Kibana --> Management --> Watchers --> edit

Match the following condition

WHEN max() OF Measurement.Value OVER all documents IS ABOVE 0.1 FOR THE LAST 1 minute

send a Mail ...

Caption: station [{{ctx.metadata.name}}] exceded limit.

{{ctx.metadata.name}} gives me the name of the watcher. But I didn't make it working showing the name of station and the values.

Thx for a bit of help.

2

You have to edit the watcher script and replace the threshold value with the context from the aggs

  "condition": {
    "script": {
      "source": "if (ctx.payload.aggregations.metricAgg.value > params.threshold) { return true; } return false;",
      "params": {
        "threshold": 0.1
      }
    }
  },
3

agreed. but somehow I dont a clue, how to implement it.

4

PUT _watcher/watch/b889a6b4-95d3-410d-af2a-c3dd3500954b
{
"trigger": {
"schedule": {
"interval": "1m"
}
},
"input": {
"search": {
"request": {
"body": {
"size": 0,
"query": {
"bool": {
"filter": {
"range": {
"Payload.Created": {
"gte": "{{ctx.trigger.scheduled_time}}||-1m",
"lte": "{{ctx.trigger.scheduled_time}}",
"format": "strict_date_optional_time||epoch_millis"
}
}
}
}
},
"aggs": {
"metricAgg": {
"max": {
"field": "Measurement.Value"
}
}
}
},
"indices": [
"myindex*"
]
}
}
},
"condition": {
"script": {
"source": "if (ctx.payload.aggregations.metricAgg.value > params.threshold) { return true; } return false;",
"params": {
"threshold": 0.1
}
}
},
"transform": {
"script": {
"source": "HashMap result = new HashMap(); result.result = ctx.payload.aggregations.metricAgg.value; return result;",
"params": {
"threshold": 0.1
}
}
},
"actions": {
"email_1": {
"email": {
"profile": "standard",
"to": [
"me@someone.com"
],
"subject": "Value Watch [{{ctx.metadata.name}}] Limit exceeded",
"body": {
"text": "Check exceeded limit 0.1"
}
}
}
}
}

5

how do I get a param not by typing (0.1) i need to get it by a value ... cause the values are dynamic....

so ....
ctx.payload.aggregations.metricAgg.value; return result;",
"params": {
"threshold": ##0.1 --> Measurement.Warning f.e.
}
...

and I need to add the location at least to the topic ... fe.

"subject": "Value Watch [{{ctx.metadata.name}}] Limit exceeded"
==> "subject": "Value Watch [{{ctx.metadata.name}}] Limit in station"+Location.Station+"exceeded"

6

I see your point, you want an email alert with the list of stations that exceed the threshold
or an email for each station ?

7

A mail per Station is the best.

8

I want to use this, to send a mail to the maintainer of the station ... so if he can see which station ... its easy ...

The guys are just to lazy to have a look at the dashboard :slight_smile:

9

Check this example may help you,
The idea is a simple scripted query that compare mesured value to warning value and generate a list of stations that should be alerted on, the foreach hit execute an action

    PUT _watcher/watch/log_event_watch
{
  "metadata": {
    "name": "IOT sensors"
  },
  "trigger": {
    "schedule": {
      "interval": "5m"
    }
  },
  "input": {
    "search": {
      "request": {
        "indices": "my-sensor-logs",
        "body": {
          "query": {
            "script": {
              "script": "doc['mesured_value'].value > doc['warning_level'].value"
            }
          }
        }
      }
    }
  },
  "condition": {
    "script": "return ctx.payload.hits.total > 1"
  },
  "actions": {
    "log_hits": {
      "foreach": "ctx.payload.hits.hits",
      "max_iterations": 500,
      "logging": {
        "text": "executed at {{ctx.execution_time}} watcher [{{ctx.metadata.name}}] and detected a seonsor with high mesure - Found id {{ctx.payload._source.stattion_id}} with value {{ctx.payload._source.mesured_value}}"
      }
    },
    "send_email": {
      "foreach": "ctx.payload.hits.hits",
      "max_iterations": 500,
      "email": {
        "to": "yassine.lasri@gmail.com",
        "subject": "Watcher Notification | Stattion {{ctx.payload._source.stattion_id}}",
        "body": "executed at {{ctx.execution_time}} watcher [{{ctx.metadata.name}}] and detected a seonsor with high mesure - Found id {{ctx.payload._source.stattion_id}} with value {{ctx.payload._source.mesured_value}}"
      }
    }
  }
}
1 Like
10

ah i tried ... but didnt work ...

I added the Indexpattern like this..."indices": "my-sensor-l*",
But it didnt fire. Maybe it's the Number of docs. To shrink the size, i think I should implement a time range in the filter ... I had something like this before:

"range": {
"Payload.Created": {
"gte": "{{ctx.trigger.scheduled_time}}||-2h",
"lte": "{{ctx.trigger.scheduled_time}}",
"format": "strict_date_optional_time||epoch_millis"
}

But I dont know where to put ...

11

Yes you can customize the search query

PUT _watcher/watch/log_event_watch
{
  "metadata": {
    "name": "IOT sensors"
  },
  "trigger": {
    "schedule": {
      "interval": "5m"
    }
  },
  "input": {
    "search": {
      "request": {
        "indices": "my-sensor-logs",
        "body": {
          "query": {
            "bool": {
              "filter": {
                "range": {
                  "@timestamp": {
                    "gte": "{{ctx.trigger.scheduled_time}}||-1m",
                    "lte": "{{ctx.trigger.scheduled_time}}",
                    "format": "strict_date_optional_time||epoch_millis"
                  }
                }
              },
              "must": [
                {
                  "script": {
                    "script": "doc['mesured_value'].value > doc['warning_level'].value"
                  }
                }
              ]
            }
          }
        }
      }
    }
  },
  "condition": {
    "script": "return ctx.payload.hits.total > 1"
  },
  "actions": {
    "log_hits": {
      "foreach": "ctx.payload.hits.hits",
      "max_iterations": 500,
      "logging": {
        "text": "executed at {{ctx.execution_time}} watcher [{{ctx.metadata.name}}] and detected a seonsor with high mesure - Found id {{ctx.payload._source.stattion_id}} with value {{ctx.payload._source.mesured_value}}"
      }
    }
  }
}
12

I'll ty this and report back asap I checked.

13

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.