# Kibana watcher help

**URL:** <https://discuss.elastic.co/t/kibana-watcher-help/323844>\
**Category:** Kibana\
**Tags:** elastic-stack-alerting\
**Created:** [January 24, 2023, 2:35pm UTC](https://discuss.elastic.co/t/kibana-watcher-help/323844 "2023-01-24T14:35:07Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![furkano](https://avatars.discourse-cdn.com/v4/letter/f/e9c0ed/32.png) [@furkano](https://discuss.elastic.co/u/furkano)\
**Post date:** [January 24, 2023, 2:35pm UTC](https://discuss.elastic.co/t/kibana-watcher-help/323844/1 "2023-01-24T14:35:07Z")

</div>

Hi,

I want to create watcher and the condition if there is same at least 10 messages in the field X for 10 minutes i need an alert,

I couldn't find any documentary for this sample, should it been done with script?

Thanks.

---

<div class="post-metadata">

**Author:** ![Ayush\_Mathur](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ayush_mathur/32/77134_2.png) [@Ayush\_Mathur](https://discuss.elastic.co/u/Ayush_Mathur)\
**Post date:** [January 24, 2023, 4:50pm UTC](https://discuss.elastic.co/t/kibana-watcher-help/323844/2 "2023-01-24T16:50:32Z")

</div>

Hello @furkano , welcome to the community !  
For your case, just a terms aggregation on fieldX with min\_doc\_count=10 should suffice. This will return a bucket of those messages which are present atleast 10 times in logs. In the query part hoever, you need to provide a range filter for timestamp field that can be `gte: now-10m` and `lte: now` for 10 mins interval.  
Once you have the buckets, you can create scripted conditions and actions as required.

---

<div class="post-metadata">

**Author:** ![furkano](https://avatars.discourse-cdn.com/v4/letter/f/e9c0ed/32.png) [@furkano](https://discuss.elastic.co/u/furkano)\
**Post date:** [January 24, 2023, 6:29pm UTC](https://discuss.elastic.co/t/kibana-watcher-help/323844/3 "2023-01-24T18:29:58Z")

</div>

Hello Ayush,

Actually i want to to this for every value in field X , not just for 1 value.  
If any value repeats itself 10 times in 10 minutes i want to get an alarm.

---

<div class="post-metadata">

**Author:** ![Ayush\_Mathur](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ayush_mathur/32/77134_2.png) [@Ayush\_Mathur](https://discuss.elastic.co/u/Ayush_Mathur)\
**Post date:** [January 25, 2023, 7:30am UTC](https://discuss.elastic.co/t/kibana-watcher-help/323844/4 "2023-01-25T07:30:48Z")

</div>

That's exactly what it's going to do, note I haven't explicitly mentioned what to query for in fieldX and I haven't provided any `include` in aggregation as well. So essentially:

1. filter will scope out the documents from last 10 mins.
2. terms aggregation will be performed on fieldX and only those messages will be included where `min_doc_count=10`, i.e. they are logged in scoped documents (from step 1) atleast 10 times.

---

<div class="post-metadata">

**Author:** ![furkano](https://avatars.discourse-cdn.com/v4/letter/f/e9c0ed/32.png) [@furkano](https://discuss.elastic.co/u/furkano)\
**Post date:** [January 25, 2023, 12:52pm UTC](https://discuss.elastic.co/t/kibana-watcher-help/323844/6 "2023-01-25T12:52:11Z")

</div>

It should be someting like below but i couldn't be sure about in the XXX part what should i add to field name

```auto
{
  "trigger": {
    "schedule": {
      "interval": "2m"
    }
  },
  "input": {
    "search": {
      "request": {
        "body": {
          "size": 0,
          "query": {
            "bool": {
              "must": [
                {
                  "match_phrase": {
                    "FIELD_NAME": "XXXXX"
                  }
                }  
              ]
          }    
        },
        "indices": [
          "INDEX_NAME"
        ]
      }
    }
  },
  "condition": {
    "compare": {
      "min_doc_count": {
        "gte": 10
      }
    }
  },
  "actions": {
    "my-logging-action": {
      "logging": {
        "text": "Threshold is 10."
       }
     }
   }
 }
}

```

---

<div class="post-metadata">

**Author:** ![Ayush\_Mathur](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ayush_mathur/32/77134_2.png) [@Ayush\_Mathur](https://discuss.elastic.co/u/Ayush_Mathur)\
**Post date:** [January 25, 2023, 1:10pm UTC](https://discuss.elastic.co/t/kibana-watcher-help/323844/7 "2023-01-25T13:10:39Z")

</div>

> [@Ayush\_Mathur](#):
>
> terms aggregation on fieldX with min\_doc\_count=10

This is what you need with range filter for timelines. What you are doing currently is a DSL query, please refer here: [Terms aggregation | Elasticsearch Guide [8.11] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-bucket-terms-aggregation.html)

---

<div class="post-metadata">

**Author:** ![furkano](https://avatars.discourse-cdn.com/v4/letter/f/e9c0ed/32.png) [@furkano](https://discuss.elastic.co/u/furkano)\
**Post date:** [January 25, 2023, 1:52pm UTC](https://discuss.elastic.co/t/kibana-watcher-help/323844/8 "2023-01-25T13:52:00Z")

</div>

This is actually working but for just a value i couldn't make it for every value that you mentioned before

```auto
{
  "trigger": {
    "schedule": {
      "interval": "5m"
    }
  },
  "input": {
    "search": {
      "request": {
        "search_type": "query_then_fetch",
        "indices": [
          "INDEX_NAME"
        ],
        "rest_total_hits_as_int": true,
        "body": {
          "query": {
            "bool": {
              "must": {
                "match": {
                  "FIELD_NAME": "VALUE_IN_FIELD"
                }
              },
              "filter": {
                "range": {
                  "@timestamp": {
                    "gte": "now-10m",
                    "lte": "now"
                  }
                }
              }
            }
          }
        }
      }
    }
  },
  "condition": {
    "compare": {
      "min_doc_count": {
        "gte": 10
      }
    }
  },
  "actions": {
    "email_1": {
      "email": {
        "profile": "standard",
        "to": [
          "MAIL_ADRESS"
        ],
        "subject": "test",
        "body": {
          "text": "test"
        }
      }
    }
  }
}

```

---

<div class="post-metadata">

**Author:** ![Ayush\_Mathur](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ayush_mathur/32/77134_2.png) [@Ayush\_Mathur](https://discuss.elastic.co/u/Ayush_Mathur)\
**Post date:** [January 25, 2023, 3:44pm UTC](https://discuss.elastic.co/t/kibana-watcher-help/323844/9 "2023-01-25T15:44:05Z")

</div>

Try something like this:

```auto
{
  "trigger": {
    "schedule": {
      "interval": "2m"
    }
  },
  "input": {
    "search": {
      "request": {
        "body": {
          "size": 0,
          "query": {
            "bool": {
              "filter": {
                  "range": {
                      "@timestamp": {
                           "gte" : "now-10m",
                           "lte": "now"
                      }
                  }
              }
          }    
        },
        "aggs": {
            "fieldX": {
                "terms" : {
                    "field": "fieldX.keyword",
                    "min_doc_count": 10
                }
            }
        },
        "indices": [
          "INDEX_NAME"
        ]
      }
    }
  },
  "condition": {
    "script": {
        "source" : "if(ctx.payload.aggregations.fieldX.buckets.length > 0) { return true;} return false;"
     }
    }
  },
  "actions": {
    "my-logging-action": {
      "logging": {
        "text": "Threshold is 10."
       }
     }
   }
 }
}

```

Please note, when you use aggregation, you need traverse through the path to aggregation buckets (similar to what we have in condition) and then access `key` (your messages or values of `fieldX` field and `doc_count` (the number of documents where "key" was present).

This must be followed in `conditions and actions` else you won't have desired results, refer: [Running an action for each element in an array | Elasticsearch Guide [8.6] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/action-foreach.html)

---

<div class="post-metadata">

**Author:** ![furkano](https://avatars.discourse-cdn.com/v4/letter/f/e9c0ed/32.png) [@furkano](https://discuss.elastic.co/u/furkano)\
**Post date:** [January 26, 2023, 11:40am UTC](https://discuss.elastic.co/t/kibana-watcher-help/323844/10 "2023-01-26T11:40:33Z")

</div>

I tried a lot of different combinations and there is still this error says "unknown input type [condition]"

My actual watcher is like this below

My field is uri,

This is my first time creating a watcher and actually i couldn't figure out how to do it

```auto
{
  "trigger": {
    "schedule": {
      "interval": "2m"
    }
  },
  "input": {
    "search": {
      "request": {
        "body": {
          "size": 0,
          "query": {
            "bool": {
              "filter": {
                  "range": {
                      "@timestamp": {
                           "gte" : "now-10m",
                           "lte": "now"
                      }
                  }
              }
          }    
        },	
        "aggs": {
            "uri": {
                "terms" : {
                    "field": "uri.keyword",
                    "min_doc_count": 10
                }
            }
        }
	},
        "indices": [
          "-logs-*"
        ] 
    }
  },
  "condition": {
    "script": {
        "source" : "if(ctx.payload.aggregations.uri.bucket.length > 0) { return true;} return false;"
     }
    }
  },
  "actions": {
    "email_1": {
      "email": {
        "profile": "standard",
        "to": [
          "@gmail.com"
        ],
        "subject": "test",
        "body": {
          "text": "test"
	  }
     }
   }
 }
}

```

---

<div class="post-metadata">

**Author:** ![Ayush\_Mathur](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ayush_mathur/32/77134_2.png) [@Ayush\_Mathur](https://discuss.elastic.co/u/Ayush_Mathur)\
**Post date:** [January 26, 2023, 11:43am UTC](https://discuss.elastic.co/t/kibana-watcher-help/323844/11 "2023-01-26T11:43:50Z")

</div>

Please check on the curly braces where they are opened and closed. Essentially every watcher has 4 components which must reside out of each other:

1. trigger schedule
2. input - query + aggs
3. condition
4. actions

In your case, it seems `condition` is inside `input {}` block. Format the JSON and check where each block is closing.

---

<div class="post-metadata">

**Author:** ![furkano](https://avatars.discourse-cdn.com/v4/letter/f/e9c0ed/32.png) [@furkano](https://discuss.elastic.co/u/furkano)\
**Post date:** [January 26, 2023, 12:07pm UTC](https://discuss.elastic.co/t/kibana-watcher-help/323844/12 "2023-01-26T12:07:42Z")

</div>

Now it's working but i couldn't be sure about the results it gave me,

Now im make some tests on it,

Thank you so much Ayush!

---

<div class="post-metadata">

**Author:** ![Ayush\_Mathur](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ayush_mathur/32/77134_2.png) [@Ayush\_Mathur](https://discuss.elastic.co/u/Ayush_Mathur)\
**Post date:** [January 26, 2023, 12:47pm UTC](https://discuss.elastic.co/t/kibana-watcher-help/323844/13 "2023-01-26T12:47:50Z")

</div>

Nice, good to know it resolved your issue 🙂

---

<div class="post-metadata">

**Author:** ![furkano](https://avatars.discourse-cdn.com/v4/letter/f/e9c0ed/32.png) [@furkano](https://discuss.elastic.co/u/furkano)\
**Post date:** [January 27, 2023, 7:38am UTC](https://discuss.elastic.co/t/kibana-watcher-help/323844/14 "2023-01-27T07:38:22Z")

</div>

Actually i've got one more request to go Ayush,

I need to add text to this watcher, it needs to show some other fields' values can it be done with something like this?

This is now workin in properyl, should i add more aggs to get the value of another field?

```auto
"{{ctx.payload.hits.total}} Errors have occured in the logs:{{#ctx.payload.hits.hits}}{{_id}}:{{/ctx.payload.hits.hits}} Result Message:{{#ctx.payload.hits.hits}}{{_source.message}}:{{/ctx.payload.hits.hits}}"

```

---

<div class="post-metadata">

**Author:** ![Ayush\_Mathur](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ayush_mathur/32/77134_2.png) [@Ayush\_Mathur](https://discuss.elastic.co/u/Ayush_Mathur)\
**Post date:** [January 27, 2023, 9:38am UTC](https://discuss.elastic.co/t/kibana-watcher-help/323844/15 "2023-01-27T09:38:08Z")

</div>

So basically you need the document ID and message logged for your errors which are aggregated ?

---

<div class="post-metadata">

**Author:** ![furkano](https://avatars.discourse-cdn.com/v4/letter/f/e9c0ed/32.png) [@furkano](https://discuss.elastic.co/u/furkano)\
**Post date:** [January 27, 2023, 10:00am UTC](https://discuss.elastic.co/t/kibana-watcher-help/323844/16 "2023-01-27T10:00:49Z")

</div>

I want almost 6-7 fields' value in my text, not just for my FieldX and they are not aggregated

---

<div class="post-metadata">

**Author:** ![Ayush\_Mathur](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ayush_mathur/32/77134_2.png) [@Ayush\_Mathur](https://discuss.elastic.co/u/Ayush_Mathur)\
**Post date:** [January 27, 2023, 10:02am UTC](https://discuss.elastic.co/t/kibana-watcher-help/323844/17 "2023-01-27T10:02:28Z")

</div>

It's kind of hard to say without looking at a sample log and sample expected output, will you be able to share those ?

---

<div class="post-metadata">

**Author:** ![furkano](https://avatars.discourse-cdn.com/v4/letter/f/e9c0ed/32.png) [@furkano](https://discuss.elastic.co/u/furkano)\
**Post date:** [January 27, 2023, 10:28am UTC](https://discuss.elastic.co/t/kibana-watcher-help/323844/18 "2023-01-27T10:28:52Z")

</div>

Here is what i want in text 🙂

Should i specify my field1, field2 or field3... in my aggregations or somewhere else?

```auto
{
  "trigger": {
    "schedule": {
      "interval": "2m"
    }
  },
  "input": {
    "search": {
      "request": {
        "search_type": "query_then_fetch",
        "indices": [
          "INDEX_NAME"
        ],
        "rest_total_hits_as_int": true,
        "body": {
          "size": 0,
          "query": {
            "bool": {
              "filter": {
                "range": {
                  "@timestamp": {
                    "gte": "now-10m",
                    "lte": "now"
                  }
                }
              }
            }
          },
          "aggs": {
            "uri": {
              "terms": {
                "field": "uri.keyword",
                "min_doc_count": 10
              }
            }
          }
        }
      }
    }
  },
  "condition": {
    "script": {
      "source": "if(ctx.payload.aggregations.uri.buckets.length > 0) { return true;} return false;",
      "lang": "painless"
    }
  },
  "actions": {
    "email_1": {
      "email": {
        "profile": "standard",
        "to": [
          "gmail"
        ],
        "subject": "ELK errors in uri field",
        "body": {
          "text": " There is an error in uri field and which is : MY VALUE IN FIELD URI,
field1 MY FIELD1'S VALUE, field2 FIELD2'S VALUE, field3 FIELD3'S value,.... "
        }
      }
    }
  }
}

```

---

<div class="post-metadata">

**Author:** ![Ayush\_Mathur](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ayush_mathur/32/77134_2.png) [@Ayush\_Mathur](https://discuss.elastic.co/u/Ayush_Mathur)\
**Post date:** [January 27, 2023, 2:03pm UTC](https://discuss.elastic.co/t/kibana-watcher-help/323844/19 "2023-01-27T14:03:27Z")

</div>

Alright, and all those fields are always present in the documents when some URI error occurs ? If not, then probably you need to put a check or add `_exists_` query in your input section.

---

<div class="post-metadata">

**Author:** ![furkano](https://avatars.discourse-cdn.com/v4/letter/f/e9c0ed/32.png) [@furkano](https://discuss.elastic.co/u/furkano)\
**Post date:** [January 28, 2023, 10:13am UTC](https://discuss.elastic.co/t/kibana-watcher-help/323844/20 "2023-01-28T10:13:08Z")

</div>

Actually fields might be exist always, but im not sure about it, im gonna ask that on monday Ayush.  
If they always exist how should i do it?

---

<div class="post-metadata">

**Author:** ![furkano](https://avatars.discourse-cdn.com/v4/letter/f/e9c0ed/32.png) [@furkano](https://discuss.elastic.co/u/furkano)\
**Post date:** [January 31, 2023, 6:25am UTC](https://discuss.elastic.co/t/kibana-watcher-help/323844/21 "2023-01-31T06:25:54Z")

</div>

Hi again Ayush, they always exist when uri field occurs. How should i make that fields come to my text in mail

[Next page](https://discuss.elastic.co/t/kibana-watcher-help/323844.md?page=2)