# Elasticsearch indexes - watcher notifications

**URL:** <https://discuss.elastic.co/t/elasticsearch-indexes-watcher-notifications/96723>\
**Category:** Elasticsearch\
**Tags:** elastic-stack-alerting\
**Created:** [August 11, 2017, 8:05am UTC](https://discuss.elastic.co/t/elasticsearch-indexes-watcher-notifications/96723 "2017-08-11T08:05:29Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![Raj\_Kumar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/raj_kumar/32/25420_2.png) [@Raj\_Kumar](https://discuss.elastic.co/u/Raj_Kumar)\
**Post date:** [August 11, 2017, 8:05am UTC](https://discuss.elastic.co/t/elasticsearch-indexes-watcher-notifications/96723/1 "2017-08-11T08:05:29Z")

</div>

Hi All,

Iam using elastic stack 5.5 to monitor the netflow and sflow and I have two separate indexes , I view them in kibana , for search and dashboards.

If i stop receiving the logs in the indexes, from logstash or the source itself stop sending the logs, in real time monitoring how can I use watcher to get notify me in email that this particular indexe is not getting events or logs ,so that I could necessary steps to see if the source is offline or my logstash got crashed.

Since I cant monitor the kibana in real time 24/7.

Any help would be really helpful  
Thanks in advance,  
Raj

---

<div class="post-metadata">

**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\
**Post date:** [August 11, 2017, 10:04am UTC](https://discuss.elastic.co/t/elasticsearch-indexes-watcher-notifications/96723/2 "2017-08-11T10:04:59Z")

</div>

hey,

you could have a watch that executes a search, that returns only the newest timestamp (either by document search or by an max aggregation). If that timestamp is older than five minutes, you could trigger an alert.

--Alex

---

<div class="post-metadata">

**Author:** ![Raj\_Kumar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/raj_kumar/32/25420_2.png) [@Raj\_Kumar](https://discuss.elastic.co/u/Raj_Kumar)\
**Post date:** [August 11, 2017, 10:29am UTC](https://discuss.elastic.co/t/elasticsearch-indexes-watcher-notifications/96723/3 "2017-08-11T10:29:56Z")

</div>

Hi Alex,

Thank you for the reply,

lets say i have a feild says , host and its value is some ip like 10.10.10.101,how do i write the watch alerts , if there is no document host is equal 10.10.10.101, in last 5 mins ?

Raj

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [August 11, 2017, 11:42am UTC](https://discuss.elastic.co/t/elasticsearch-indexes-watcher-notifications/96723/4 "2017-08-11T11:42:41Z")

</div>

The watch you are looking for sounds similar in structure to the [example provided here](https://github.com/elastic/examples/tree/master/Alerting/Sample%20Watches/system_fails_to_provide_data).

---

<div class="post-metadata">

**Author:** ![Raj\_Kumar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/raj_kumar/32/25420_2.png) [@Raj\_Kumar](https://discuss.elastic.co/u/Raj_Kumar)\
**Post date:** [August 11, 2017, 1:27pm UTC](https://discuss.elastic.co/t/elasticsearch-indexes-watcher-notifications/96723/5 "2017-08-11T13:27:08Z")

</div>

Hi Dave,

Thank you for the reply,when am trying to execute it says

---

<div class="post-metadata">

**Author:** ![devans](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/devans/32/21073_2.png) [@devans](https://discuss.elastic.co/u/devans)\
**Post date:** [August 11, 2017, 3:26pm UTC](https://discuss.elastic.co/t/elasticsearch-indexes-watcher-notifications/96723/6 "2017-08-11T15:26:16Z")

</div>

You need to define the scripts in someway as described [here](https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-scripting-using.html).

Or put insert them into the watch, which isn't as graceful, but works for testing. I just did something similar with the same watch as a starting point.

```auto
"condition": {
    "script": {
      "inline": "return ctx.payload.hosts.aggregations.hosts.buckets.size() != ctx.payload.history_hosts.aggregations.hosts.buckets.size();",
      "lang": "painless"
    }
  },

```

---

<div class="post-metadata">

**Author:** ![Raj\_Kumar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/raj_kumar/32/25420_2.png) [@Raj\_Kumar](https://discuss.elastic.co/u/Raj_Kumar)\
**Post date:** [August 14, 2017, 8:44am UTC](https://discuss.elastic.co/t/elasticsearch-indexes-watcher-notifications/96723/7 "2017-08-14T08:44:22Z")

</div>

Hi Dave,

Thank you for the reply,when am trying to execute it says

"exception": {  
"type": "script\_exception",  
"reason": "runtime error",  
"script\_stack": [  
"return ctx.payload.hosts.aggregations.hosts.buckets.size() != ctx.payload.history\_hosts.aggregations.hosts.buckets.size();",  
" ^---- HERE"  
],  
"script": "return ctx.payload.hosts.aggregations.hosts.buckets.size() != ctx.payload.history\_hosts.aggregations.hosts.buckets.size();",  
"lang": "painless",  
"caused\_by": {  
"type": "null\_pointer\_exception",  
"reason": null,  
"stack\_trace": "java.lang.NullPointerException\n\tat org.elasticsearch.painless.DefBootstrap$PIC.fallback(DefBootstrap.java:202)\n\tat org.elasticsearch.painless.PainlessScript$Script.execute(return ctx.payload.hosts.aggregations.hosts.buckets.size() != ctx.payload.history\_hosts.aggregations.hosts.buckets.size();:25)\n\tat org.elasticsearch.painless.ScriptImpl.run(ScriptImpl.java:133)\n\tat org.elasticsearch.xpack.watcher.condition.ScriptCondition.doExecute(ScriptCondition.java:94)\n\tat org.elasticsearch.xpack.watcher.condition.ScriptCondition.execute(ScriptCondition.java:84)\n\tat org.elasticsearch.xpack.watcher.execution.ExecutionService.executeInner(ExecutionService.java:391)\n\tat org.elasticsearch.xpack.watcher.execution.ExecutionService.execute(ExecutionService.java:275)\n\tat org.elasticsearch.xpack.watcher.execution.ExecutionService$WatchExecutionTask.run(ExecutionService.java:492)\n\tat org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:569)\n\tat java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)\n\tat java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)\n\tat java.lang.Thread.run(Thread.java:748)\n"  
},  
"stack\_trace": "ScriptException[runtime error]; nested: NullPointerException;\n\tat org.elasticsearch.painless.PainlessScript.convertToScriptException(PainlessScript.java:101)\n\tat org.elasticsearch.painless.PainlessScript$Script.execute(return ctx.payload.hosts.aggregations.hosts.buckets.size() != ctx.payload.history\_hosts.aggregations.hosts.buckets.size();:8)\n\tat org.elasticsearch.painless.ScriptImpl.run(ScriptImpl.java:133)\n\tat org.elasticsearch.xpack.watcher.condition.ScriptCondition.doExecute(ScriptCondition.java:94)\n\tat org.elasticsearch.xpack.watcher.condition.ScriptCondition.execute(ScriptCondition.java:84)\n\tat org.elasticsearch.xpack.watcher.execution.ExecutionService.executeInner(ExecutionService.java:391)\n\tat org.elasticsearch.xpack.watcher.execution.ExecutionService.execute(ExecutionService.java:275)\n\tat org.elasticsearch.xpack.watcher.execution.ExecutionService$WatchExecutionTask.run(ExecutionService.java:492)\n\tat org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:569)\n\tat java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)\n\tat java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)\n\tat java.lang.Thread.run(Thread.java:748)\nCaused by: java.lang.NullPointerException\n\tat org.elasticsearch.painless.DefBootstrap$PIC.fallback(DefBootstrap.java:202)\n\tat org.elasticsearch.painless.PainlessScript$Script.execute(return ctx.payload.hosts.aggregations.hosts.buckets.size() != ctx.payload.history\_hosts.aggregations.hosts.buckets.size();:25)\n\t... 10 more\n"  
}  
}

---

<div class="post-metadata">

**Author:** ![devans](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/devans/32/21073_2.png) [@devans](https://discuss.elastic.co/u/devans)\
**Post date:** [August 15, 2017, 11:43am UTC](https://discuss.elastic.co/t/elasticsearch-indexes-watcher-notifications/96723/8 "2017-08-15T11:43:10Z")

</div>

The example I gave had different aggregation names.

```auto
"condition": {
    "script": {
      "inline": "return ctx.payload.aggregations.periods.buckets.history.hosts.buckets.size() > ctx.payload.aggregations.periods.buckets.last_period.hosts.buckets.size();",
      "lang": "painless"
    }
  },

```

---

<div class="post-metadata">

**Author:** ![Raj\_Kumar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/raj_kumar/32/25420_2.png) [@Raj\_Kumar](https://discuss.elastic.co/u/Raj_Kumar)\
**Post date:** [August 18, 2017, 12:30pm UTC](https://discuss.elastic.co/t/elasticsearch-indexes-watcher-notifications/96723/9 "2017-08-18T12:30:55Z")

</div>

Thank you so muck for the reply 🙂

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 15, 2017, 12:31pm UTC](https://discuss.elastic.co/t/elasticsearch-indexes-watcher-notifications/96723/10 "2017-09-15T12:31:15Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.