Hi,
I want to create watcher and the condition if there is same at least 10 messages in the field X for 10 minutes i need an alert,
I couldn't find any documentary for this sample, should it been done with script?
Thanks.
Hello @furkano , welcome to the community !
For your case, just a terms aggregation on fieldX with min_doc_count=10 should suffice. This will return a bucket of those messages which are present atleast 10 times in logs. In the query part hoever, you need to provide a range filter for timestamp field that can be gte: now-10m and lte: now for 10 mins interval.
Once you have the buckets, you can create scripted conditions and actions as required.
Hello Ayush,
Actually i want to to this for every value in field X , not just for 1 value.
If any value repeats itself 10 times in 10 minutes i want to get an alarm.
That's exactly what it's going to do, note I haven't explicitly mentioned what to query for in fieldX and I haven't provided any include in aggregation as well. So essentially:
- filter will scope out the documents from last 10 mins.
- terms aggregation will be performed on fieldX and only those messages will be included where
min_doc_count=10, i.e. they are logged in scoped documents (from step 1) atleast 10 times.
It should be someting like below but i couldn't be sure about in the XXX part what should i add to field name
{
"trigger": {
"schedule": {
"interval": "2m"
}
},
"input": {
"search": {
"request": {
"body": {
"size": 0,
"query": {
"bool": {
"must": [
{
"match_phrase": {
"FIELD_NAME": "XXXXX"
}
}
]
}
},
"indices": [
"INDEX_NAME"
]
}
}
},
"condition": {
"compare": {
"min_doc_count": {
"gte": 10
}
}
},
"actions": {
"my-logging-action": {
"logging": {
"text": "Threshold is 10."
}
}
}
}
}
This is what you need with range filter for timelines. What you are doing currently is a DSL query, please refer here: Terms aggregation | Elasticsearch Guide [8.11] | Elastic
This is actually working but for just a value i couldn't make it for every value that you mentioned before
{
"trigger": {
"schedule": {
"interval": "5m"
}
},
"input": {
"search": {
"request": {
"search_type": "query_then_fetch",
"indices": [
"INDEX_NAME"
],
"rest_total_hits_as_int": true,
"body": {
"query": {
"bool": {
"must": {
"match": {
"FIELD_NAME": "VALUE_IN_FIELD"
}
},
"filter": {
"range": {
"@timestamp": {
"gte": "now-10m",
"lte": "now"
}
}
}
}
}
}
}
}
},
"condition": {
"compare": {
"min_doc_count": {
"gte": 10
}
}
},
"actions": {
"email_1": {
"email": {
"profile": "standard",
"to": [
"MAIL_ADRESS"
],
"subject": "test",
"body": {
"text": "test"
}
}
}
}
}
Try something like this:
{
"trigger": {
"schedule": {
"interval": "2m"
}
},
"input": {
"search": {
"request": {
"body": {
"size": 0,
"query": {
"bool": {
"filter": {
"range": {
"@timestamp": {
"gte" : "now-10m",
"lte": "now"
}
}
}
}
},
"aggs": {
"fieldX": {
"terms" : {
"field": "fieldX.keyword",
"min_doc_count": 10
}
}
},
"indices": [
"INDEX_NAME"
]
}
}
},
"condition": {
"script": {
"source" : "if(ctx.payload.aggregations.fieldX.buckets.length > 0) { return true;} return false;"
}
}
},
"actions": {
"my-logging-action": {
"logging": {
"text": "Threshold is 10."
}
}
}
}
}
Please note, when you use aggregation, you need traverse through the path to aggregation buckets (similar to what we have in condition) and then access key (your messages or values of fieldX field and doc_count (the number of documents where "key" was present).
This must be followed in conditions and actions else you won't have desired results, refer: Running an action for each element in an array | Elasticsearch Guide [8.6] | Elastic
I tried a lot of different combinations and there is still this error says "unknown input type [condition]"
My actual watcher is like this below
My field is uri,
This is my first time creating a watcher and actually i couldn't figure out how to do it
{
"trigger": {
"schedule": {
"interval": "2m"
}
},
"input": {
"search": {
"request": {
"body": {
"size": 0,
"query": {
"bool": {
"filter": {
"range": {
"@timestamp": {
"gte" : "now-10m",
"lte": "now"
}
}
}
}
},
"aggs": {
"uri": {
"terms" : {
"field": "uri.keyword",
"min_doc_count": 10
}
}
}
},
"indices": [
"-logs-*"
]
}
},
"condition": {
"script": {
"source" : "if(ctx.payload.aggregations.uri.bucket.length > 0) { return true;} return false;"
}
}
},
"actions": {
"email_1": {
"email": {
"profile": "standard",
"to": [
"@gmail.com"
],
"subject": "test",
"body": {
"text": "test"
}
}
}
}
}
Please check on the curly braces where they are opened and closed. Essentially every watcher has 4 components which must reside out of each other:
- trigger schedule
- input - query + aggs
- condition
- actions
In your case, it seems condition is inside input {} block. Format the JSON and check where each block is closing.
Now it's working but i couldn't be sure about the results it gave me,
Now im make some tests on it,
Thank you so much Ayush!
1 Like
Nice, good to know it resolved your issue 
Actually i've got one more request to go Ayush,
I need to add text to this watcher, it needs to show some other fields' values can it be done with something like this?
This is now workin in properyl, should i add more aggs to get the value of another field?
"{{ctx.payload.hits.total}} Errors have occured in the logs:{{#ctx.payload.hits.hits}}{{_id}}:{{/ctx.payload.hits.hits}} Result Message:{{#ctx.payload.hits.hits}}{{_source.message}}:{{/ctx.payload.hits.hits}}"
So basically you need the document ID and message logged for your errors which are aggregated ?
I want almost 6-7 fields' value in my text, not just for my FieldX and they are not aggregated
It's kind of hard to say without looking at a sample log and sample expected output, will you be able to share those ?
Here is what i want in text
Should i specify my field1, field2 or field3... in my aggregations or somewhere else?
{
"trigger": {
"schedule": {
"interval": "2m"
}
},
"input": {
"search": {
"request": {
"search_type": "query_then_fetch",
"indices": [
"INDEX_NAME"
],
"rest_total_hits_as_int": true,
"body": {
"size": 0,
"query": {
"bool": {
"filter": {
"range": {
"@timestamp": {
"gte": "now-10m",
"lte": "now"
}
}
}
}
},
"aggs": {
"uri": {
"terms": {
"field": "uri.keyword",
"min_doc_count": 10
}
}
}
}
}
}
},
"condition": {
"script": {
"source": "if(ctx.payload.aggregations.uri.buckets.length > 0) { return true;} return false;",
"lang": "painless"
}
},
"actions": {
"email_1": {
"email": {
"profile": "standard",
"to": [
"gmail"
],
"subject": "ELK errors in uri field",
"body": {
"text": " There is an error in uri field and which is : MY VALUE IN FIELD URI,
field1 MY FIELD1'S VALUE, field2 FIELD2'S VALUE, field3 FIELD3'S value,.... "
}
}
}
}
}
Alright, and all those fields are always present in the documents when some URI error occurs ? If not, then probably you need to put a check or add _exists_ query in your input section.
Actually fields might be exist always, but im not sure about it, im gonna ask that on monday Ayush.
If they always exist how should i do it?
Hi again Ayush, they always exist when uri field occurs. How should i make that fields come to my text in mail