But this only returns the message field of one error. But this is not my requirement.
For ex: If I have 7 different errors in my logs, I want to retrieve the "message" field for all these 7 errors. Similarly, if I have 100 logs of level "ERROR", Watcher should send me the "message" field of each of these 100 errors.
Please find below the script that I have written in order to achieve this.
I also have other inputs as a part of the input chain. But I haven't included them because they weren't of any concern. Will I have to use aggregations in order to achieve what I'm trying to achieve?
Thank you so much for your help. It works perfectly now!!
I used the following snippet in order to achieve this:
{
"trigger": {
"schedule": {
"interval": "10h"
}
},
"input": {
"chain": {
"inputs": [
{
"first": {
"search": {
"request": {
"search_type": "query_then_fetch",
"indices": [
"logs"
],
"types": [],
"body": {
"query": {
"constant_score": {
"filter": {
"bool": {
"filter": [
{
"range": {
"@timestamp": {
"gte": "now-24h"
}
}
}
],
"must": [
{
"match": {
"level": "ERROR"
}
},
{
"bool": {
"must": [
{
"match": {
"host": "some IP address"
}
}
]
}
}
]
}
}
}
},
"_source": [
"message"
]
}
}
}
}
},
"condition": {
"compare": {
"ctx.payload.first.hits.total": {
"gt": 15
}
}
},
"actions": {
"send_email": {
"email": {
"profile": "standard",
"to": [ "aa@bb.com"
],
"subject": "ERRORS in your specified host are exceeding a count of 15 - Errors that occurred over the past 24 hours",
"body": {
"text": "{{ctx.payload.first.hits.total}} errors have occurred in your specified host \n\n The errors that have occurred over the past 24 hours can be found below:\n\n {{#ctx.payload.first.hits.hits}}{{_source}}:\n{{/ctx.payload.first.hits.hits}}
}
}
}
}
}
I also found that I could retrieve other components from my logs by simply appending their tags as comma separated values in the "_source": [ ] block.
For example: "_source" : [ "message", "@timestamp" ] would give me both the messages of the errors in my system and also their corresponding time stamps.
the time zone parameter is only applied on query time. If the original time is part of your JSON document, you will be able to access it via the hits array in your payload.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.