# Creating watcher script if particular value ina field occurs more than five times in five minutes interval of time

**URL:** <https://discuss.elastic.co/t/creating-watcher-script-if-particular-value-ina-field-occurs-more-than-five-times-in-five-minutes-interval-of-time/259340>\
**Category:** Kibana\
**Tags:** elastic-stack-alerting\
**Created:** [December 22, 2020, 7:47am UTC](https://discuss.elastic.co/t/creating-watcher-script-if-particular-value-ina-field-occurs-more-than-five-times-in-five-minutes-interval-of-time/259340 "2020-12-22T07:47:56Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![lokeshbabloo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lokeshbabloo/32/81283_2.png) [@lokeshbabloo](https://discuss.elastic.co/u/lokeshbabloo)\
**Post date:** [December 22, 2020, 7:47am UTC](https://discuss.elastic.co/t/creating-watcher-script-if-particular-value-ina-field-occurs-more-than-five-times-in-five-minutes-interval-of-time/259340/1 "2020-12-22T07:47:56Z")

</div>

Hi can anyone help how can we create a watcher script for the below requirement

I having a field called "log\_message" contaning the below message

Failed to load TermRecord for TermID=

here the terminal id can be any characters , it may be number , alphabets or special characters

i want to get an alert in kibana, if the same log message containing the same terminal id occurs more than 5 times in a 5 minute interval of time. i dont know how to aggregate these in the watcher script. below is my watcher script.i used the regex pattern in the "query string" as the terminal id can be anything. but i am getting an error. is there any other way? Thanks.

```
            {
             "trigger": {
             "schedule": {
                  "interval": "5m"
                                     }
                                   },
                     "input": {
                               "search": {
                                      "request": {
                                  "search_type": "query_then_fetch",
                                  "indices": [
                 "testingalert*"
                                        ],
                                  "rest_total_hits_as_int": true,
                        "body": {
                                         "size": 0,
                                       "query": {
                                    "bool": {
                                                 "must": {
                       "query_string": {
                                 "query": "Failed to load TermRecord for TermID=^[a-zA-Z0-9_.-]*$"
                                }
                                    },
                                  "filter": {
                     "range": {
                       "@timestamp": {
                         "gte": "{{ctx.trigger.scheduled_time}}||-5m",
                           "lte": "{{ctx.trigger.scheduled_time}}",
                          "format": "strict_date_optional_time||epoch_millis"
                                }
                             }
                             }
                             }
                                },
                                  "aggs": {
                               "bucketAgg": {
                             "terms": {
                                    "field": "log_message.keyword",
                               "size": "5",
                                   "order": {
                              "_count": "desc"
                                      }
                                           }
                                    }
                                  }
                                            }
                                   }
                                            }
                          },

```

"condition": {  
"script": {  
"source": "ArrayList arr = ctx.payload.aggregations.bucketAgg.buckets; for (int i = 0; i \< arr.length; i++) { if (arr[i].doc\_count \> params.threshold) { return true; } } return false;",  
"lang": "painless",  
"params": {  
"threshold": 5  
}  
}  
},  
"actions": {  
"send\_email": {  
"email": {  
"profile": "standard",  
"to": [  
"[anonymousbeendetected@gmail.com](mailto:anonymousbeendetected@gmail.com)"  
],  
"subject": "Watcher Notification",  
"body": {  
"text": "Watch [{{ctx.metadata.name}}] The 'Warning' alert has occured more than 5 times in 5 minutes interval of time"  
}  
}  
}  
},  
"transform": {  
"script": {  
"source": "HashMap result = new HashMap(); ArrayList arr = ctx.payload.aggregations.bucketAgg.buckets; ArrayList filteredHits = new ArrayList(); for (int i = 0; i \< arr.length; i++) { HashMap filteredHit = new HashMap(); filteredHit.key = arr[i].key; filteredHit.value = arr[i].doc\_count; if (filteredHit.value \> params.threshold) { filteredHits.add(filteredHit); } } result.results = filteredHits; return result;",  
"lang": "painless",  
"params": {  
"threshold": 5  
}  
}  
}  
}

---

<div class="post-metadata">

**Author:** ![RamyaGowda](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ramyagowda/32/27440_2.png) [@RamyaGowda](https://discuss.elastic.co/u/RamyaGowda)\
**Post date:** [December 23, 2020, 5:44am UTC](https://discuss.elastic.co/t/creating-watcher-script-if-particular-value-ina-field-occurs-more-than-five-times-in-five-minutes-interval-of-time/259340/2 "2020-12-23T05:44:02Z")

</div>

Hi,

for this use case an aggregation would be as below, which checks last 5 mins data and aggregate based on message and term ID

> {  
> "size": 0,  
> "query": {  
> "range": {  
> "Timefield": {  
> "lte": "now-5m"  
> }  
> }  
> },  
> "aggs": {  
> "message": {  
> "terms": {  
> "field": "message.keyword",  
> "size": 10  
> },  
> "aggs": {  
> "id": {  
> "terms": {  
> "field": "term\_id.keyword",  
> "size": 10  
> }  
> }  
> }  
> }  
> }  
> }

And the Result looks as follows , wherein you have to loop through the payload and apply condition on the "doc\_count", if(doc\_count \>= 5) trigger an action

From my example for Log message "Time Out" Term ID "1" and "8()(#$%" are appearing more than 5 times .

> "aggregations" : {  
> "message" : {  
> "doc\_count\_error\_upper\_bound" : 0,  
> "sum\_other\_doc\_count" : 0,  
> "buckets" : [  
> {  
> **"key" : "Time Out"** ,  
> "doc\_count" : 13,  
> "id" : {  
> "doc\_count\_error\_upper\_bound" : 0,  
> "sum\_other\_doc\_count" : 0,  
> "buckets" : [  
> {  
> **"key" : "1",**  
> **"doc\_count" : 6**  
> },  
> {  
> **"key" : "8()(#$%",**  
> **"doc\_count" : 6**  
> },  
> {  
> "key" : "abc",  
> "doc\_count" : 1  
> }  
> ]  
> }  
> }  
> ]  
> }  
> }

Hope my explanation is clear

Thanks,  
Ramya

---

<div class="post-metadata">

**Author:** ![lokeshbabloo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lokeshbabloo/32/81283_2.png) [@lokeshbabloo](https://discuss.elastic.co/u/lokeshbabloo)\
**Post date:** [December 23, 2020, 6:02am UTC](https://discuss.elastic.co/t/creating-watcher-script-if-particular-value-ina-field-occurs-more-than-five-times-in-five-minutes-interval-of-time/259340/3 "2020-12-23T06:02:08Z")

</div>

Hi Ramya,

Thank you for your reply. By the way how will be my final watcher looks like which i have mentioned in the previous message? when i tested in my dev tools i am getting error like this

{  
"error" : {  
"root\_cause" : [  
{  
"type" : "parsing\_exception",  
"reason" : "[range] query does not support [tte]",  
"line" : 6,  
"col" : 8  
}  
],  
"type" : "parsing\_exception",  
"reason" : "[range] query does not support [tte]",  
"line" : 6,  
"col" : 8  
},  
"status" : 400  
}

can you help me on this watcher script?

Thanks  
M.Lokesh

---

<div class="post-metadata">

**Author:** ![RamyaGowda](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ramyagowda/32/27440_2.png) [@RamyaGowda](https://discuss.elastic.co/u/RamyaGowda)\
**Post date:** [December 23, 2020, 6:20am UTC](https://discuss.elastic.co/t/creating-watcher-script-if-particular-value-ina-field-occurs-more-than-five-times-in-five-minutes-interval-of-time/259340/4 "2020-12-23T06:20:09Z")

</div>

There was a typo in the query which i have corrected.  
This query & aggregation are input to watcher , you have to work on condition, transform and action.

Thanks,  
Ramya

---

<div class="post-metadata">

**Author:** ![lokeshbabloo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lokeshbabloo/32/81283_2.png) [@lokeshbabloo](https://discuss.elastic.co/u/lokeshbabloo)\
**Post date:** [December 23, 2020, 6:23am UTC](https://discuss.elastic.co/t/creating-watcher-script-if-particular-value-ina-field-occurs-more-than-five-times-in-five-minutes-interval-of-time/259340/5 "2020-12-23T06:23:30Z")

</div>

Hi Ramya,

Yeah got it. Thank you so much 😇

---

<div class="post-metadata">

**Author:** ![lokeshbabloo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lokeshbabloo/32/81283_2.png) [@lokeshbabloo](https://discuss.elastic.co/u/lokeshbabloo)\
**Post date:** [December 23, 2020, 9:13am UTC](https://discuss.elastic.co/t/creating-watcher-script-if-particular-value-ina-field-occurs-more-than-five-times-in-five-minutes-interval-of-time/259340/6 "2020-12-23T09:13:26Z")

</div>

Hi @RamyaGowda i have few doubts caan you clear that?

Failed to load TermRecord for TermID= /a-zA-Z0-9\_.-/

is this regex pattern correct? di i need to enable any regex option in the elasticsearch.yml file ? and what is the "size" in aggregation means?

---

<div class="post-metadata">

**Author:** ![RamyaGowda](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ramyagowda/32/27440_2.png) [@RamyaGowda](https://discuss.elastic.co/u/RamyaGowda)\
**Post date:** [December 23, 2020, 12:12pm UTC](https://discuss.elastic.co/t/creating-watcher-script-if-particular-value-ina-field-occurs-more-than-five-times-in-five-minutes-interval-of-time/259340/7 "2020-12-23T12:12:03Z")

</div>

Are you trying to match any character using regex ? I believe no need to do this, term aggregation based the type keyword would solve your purpose.

size is the number of documents you want to aggregate , default is 10 you can change it.

> {  
> "size": 0,  
> "aggs": {  
> "message": {  
> "terms": {  
> "field": "message.keyword",  
> "size": 10  
> },  
> "aggs": {  
> "id": {  
> "terms": {  
> "field": "term\_id.keyword",  
> "size": 1  
> }  
> }  
> }  
> }  
> }  
> }
> 
> "aggregations" : {  
> "message" : {  
> "doc\_count\_error\_upper\_bound" : 0,  
> "sum\_other\_doc\_count" : 0,  
> "buckets" : [  
> {  
> "key" : "Time Out",  
> "doc\_count" : 17,  
> "id" : {  
> "doc\_count\_error\_upper\_bound" : 0,  
> "sum\_other\_doc\_count" : 11,  
> "buckets" : [  
> {  
> "key" : "1",  
> "doc\_count" : 6  
> }  
> ]  
> }  
> }  
> ]  
> }  
> }

---

<div class="post-metadata">

**Author:** ![lokeshbabloo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lokeshbabloo/32/81283_2.png) [@lokeshbabloo](https://discuss.elastic.co/u/lokeshbabloo)\
**Post date:** [December 23, 2020, 12:52pm UTC](https://discuss.elastic.co/t/creating-watcher-script-if-particular-value-ina-field-occurs-more-than-five-times-in-five-minutes-interval-of-time/259340/8 "2020-12-23T12:52:24Z")

</div>

if suppose i am having a log like this

**\<#\> 20200806 17:04:24.834 282005997 FD.INF [MVINB-LSL2.T1G1\_ATMTH1 main.main DAPoolManager.DAControl] Failed to load TermRecord for TermID=bafldd3,MerchantID=gcl6upi**

So what ever values comes after "Term Record for" whether it may be term id or merchant id, etc . what the query will be like for in this condition?  
And thank you so much for the previous script its working perfectly fine and i am able to detect the alert 🙂

If the condition for the above requirement is achieved, my entire requirement will be resolved. can you help me on this doubt alone please?

---

<div class="post-metadata">

**Author:** ![lokeshbabloo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lokeshbabloo/32/81283_2.png) [@lokeshbabloo](https://discuss.elastic.co/u/lokeshbabloo)\
**Post date:** [December 24, 2020, 10:06am UTC](https://discuss.elastic.co/t/creating-watcher-script-if-particular-value-ina-field-occurs-more-than-five-times-in-five-minutes-interval-of-time/259340/9 "2020-12-24T10:06:24Z")

</div>

@RamyaGowda is there a way for this requirement?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [January 21, 2021, 10:06am UTC](https://discuss.elastic.co/t/creating-watcher-script-if-particular-value-ina-field-occurs-more-than-five-times-in-five-minutes-interval-of-time/259340/10 "2021-01-21T10:06:35Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.