Creating Kibana watcher if two field values has been occured more than 2 times in a given interval of time

Hi ,

Can anyone please help me on this scenario to create an alert
I am having a log events like the following

<#> 20200806 17:04:23.261 280018000 EV.WRN [ MVINB-LSL2.T1G1_WEBSVR1 main.main TSP1.T1G1_WEBSVR1 ] Failed to load TermRecord for TermID=f6y4jkm

<#> 20200806 17:04:23.261 280018000 EV.WRN [ MVINB-LSL2.T1G1_WEBSVR1 main.main TSP1.T1G1_WEBSVR1 ] Failed to load TermRecord for TermID=f6y4jkm

<#> 20200806 17:04:23.261 280018000 EV.INF [ MVINB-LSL2.T1G1_WEBSVR1 main.main TSP1.T1G1_WEBSVR1 ] Failed to load TermRecord for TermID=7yhe73v

<#> 20200806 17:04:23.261 280018000 EV.FNE [ MVINB-LSL2.T1G1_WEBSVR1 main.main TSP1.T1G1_WEBSVR1 ] Failed to load TermRecord for TermID=j6ljudn

here EV.WRN is the "event_type" and "Failed to load TermRecord for TermID=f6y4jkm" is the log_message.

i want to create an alert if get the the same "event_type" with the same "log_message"
with the same term id. Is there a way to generate and alert based upon this scenario. the following is my current watcher script.

{
   "trigger":{
      "schedule":{
         "interval":"10s"
      }
   },
   "input":{
      "search":{
         "request":{
            "search_type":"query_then_fetch",
            "indices":[
               "testingalert*"
            ],
            "rest_total_hits_as_int":true,
            "body":{
               "size":0,
               "query":{
                  "bool":{
                     "must":{
                        "query_string":{
                           "query":"Failed to load TermRecord for TermID=/a-zA-Z0-9_.-/"
                        }
                     },
                     "filter":{
                        "range":{
                           "@timestamp":{
                              "gte":"{{ctx.trigger.scheduled_time}}||-5m",
                              "lte":"{{ctx.trigger.scheduled_time}}",
                              "format":"strict_date_optional_time||epoch_millis"
                           }
                        }
                     }
                  }
               },
               "aggs":{
                  "log_message":{
                     "terms":{
                        "field":"log_message.keyword",
                        "size":10
                     },
                     "aggs":{
                        "id":{
                           "terms":{
                              "field":"term_id.keyword",
                              "size":10
                           }
                        }
                     }
                  }
               }
            }
         }
      }
   },
   "condition":{
      "script":{
         "source":"ArrayList arr = ctx.payload.aggregations.log_message.buckets; for (int i = 0; i < arr.length; i++) { if (arr[i].doc_count > params.threshold) { return true; } } return false;",
         "lang":"painless",
         "params":{
            "threshold":2
         }
      }
   },
   "actions":{
      "send_email":{
         "email":{
            "profile":"standard",
            "to":[
               "redacted"
            ],
            "subject":"Watcher Notification",
            "body":{
               "text":"Watch [{{ctx.metadata.name}}] The 'Terminal ID' alert has occured more than 5 times in 5 minutes interval of time"
            }
         }
      }
   },
   "transform":{
      "script":{
         "source":"HashMap result = new HashMap(); ArrayList arr = ctx.payload.aggregations.log_message.buckets; ArrayList filteredHits = new ArrayList(); for (int i = 0; i < arr.length; i++) { HashMap filteredHit = new HashMap(); filteredHit.key = arr[i].key; filteredHit.value = arr[i].doc_count; if (filteredHit.value > params.threshold) { filteredHits.add(filteredHit); } } result.results = filteredHits; return result;",
         "lang":"painless",
         "params":{
            "threshold":2
         }
      }
   }
}

Hi lokeshbabloo,

Same, compared to what? Based on the title, I assume, you're looking for the predicate of maxCount > 1 where maxCount is a max aggregation of a count aggregation grouped by {event_type, log_message, TermID} is it the correct understanding?

1 Like

Hi @monfera ,

Thank you for your reply . yes exactly, when grouped by { event_type, log_message, TermID } .

for example if i am getting the same term id number with the same event type, need to trigger alert. the below screenshot is my sample log. in the below screenshot you can see my term id number and my event type are same.

Thanks
Lokesh.

HI ELK stack , please close this topic as i was unable to delete the post. My time has been exceeded the limit.

Thanks and regards,
Lokesh