Alerting with a chain-input

Hi,

We are getting NPS logs and want to monitor when a certain string appears. We want to create an alert if we receive a log containing the field "CISCOTAG" containing the value "%ASA-2-113022". But, we only want to send an alert for that log, if there has been no "up" log for that particular device in 15 minutes.

So here is what i have so far:

A chain input where the first search gets all documents containing the value "%ASA-2-113022", this is the "down" event. And a second input which gets all documents containing the value "%ASA-2-113023", which is the "up" event.

    {
      "trigger": {
        "schedule": {
          "interval": "10m"
        }
      },
      "input" : {
      "chain" : {
        "inputs" : [ 
          {
            "first" : {
             "search" : {
                "request" : {
                "indices" : [ "index-*" ],
                "body" : {
                    "query": {
                        "bool": {
                        "must": [
                        {
                            "match": {
                            "ciscotag": "ASA-2-113022"
                            }
                        }
                        ],
                        "filter": {
                            "range": {
                                "@timestamp": {
                                "gte": "now-24h"
                                }
                            }
                        }
                        }
                    }
                }
                }
            }
            }
          },
          {
            "second" : {
              "search" : {
                "request" : {
                "indices" : [ "index-*" ],
                "body" : {
                    "query": {
                        "bool": {
                        "must": [
                        {
                            "match": {
                            "ciscotag": "ASA-2-113023"
                            }
                        }
                        ],
                        "filter": {
                            "range": {
                                "@timestamp": {
                                "gte": "now-24h"
                                }
                            }
                        }
                        }
                    }
                }
                }
            }
            }
          }
        ]
      }
    },
      "condition": {
        "compare": {
          "ctx.payload.hits.total": {
            "gte": 1
          }
        }
      },
      "actions": {
        "send_email": {
          "email": {
            "profile": "standard",
            "to": [
              "administrator@example.com"
            ],
            "subject": "Alert",
            "body": {
              "text": "Test"
            }
          }
        }
      }
    }

However, i'm quite stuck right now. I don't understand what to put in the condition field. I would appreciate any help

Hey,

how about a terms aggregation on the field that contains the up and down status, so that you can check if there is more than one state. This way you could also do this with a single search request.

It would help a lot if you could also share the design of your documents, as this is nearly impossible to tell form the information provided, but crucial to create a good solution.

Thanks!

--Alex

Hi,

Here is an example of the "down" log:

{
  "_index": "index-2020-05",
  "_type": "_doc",
  "_id": "UNX4B3IB0Bonuusr9wbF",
  "_version": 1,
  "_score": null,
  "_source": {
    "tags": [
      "syslog",
      "sql_successful",
      "translated",
      "iasdasd",
      "_geoip_lookup_failure",
      "monthly"
    ],
    "message": "<162>May 12 2020 10:20:54: %ASA-2-113022: AAA Marking RADIUS server 127.0.0.1 in aaa-server group RANDOMGROUP as FAILED\n",
    "@version": "1",
    "ServiceName": "service-x",
    "syslog_pri": "162",
    "host": "127.0.0.14",
    "ciscotag": "ASA-2-113022",
    "@timestamp": "2020-05-12T08:20:54.108Z",
    "unique_hostname": "hostname.example.com",
    "SID": "s112731",
    "timestamp": "May 12 2020 10:20:54"
  },
  "fields": {
    "@timestamp": [
      "2020-05-12T08:20:54.108Z"
    ]
  },
  "highlight": {
    "ciscotag": [
      "@kibana-highlighted-field@ASA@/kibana-highlighted-field@-@kibana-highlighted-field@2@/kibana-highlighted-field@-@kibana-highlighted-field@113022@/kibana-highlighted-field@"
    ]
  },
  "sort": [
    1589271654108
  ]
}

And here is an example of the "up" log:

{
  "_index": "index-2020-05",
  "_type": "_doc",
  "_id": "UNX4B3IB0Bonuusr9wbF",
  "_version": 1,
  "_score": null,
  "_source": {
    "tags": [
      "syslog",
      "sql_successful",
      "translated",
      "iasdasd",
      "_geoip_lookup_failure",
      "monthly"
    ],
    "message": "<162>May 12 2020 10:22:02: %ASA-2-113023: AAA Marking RADIUS server 127.0.0.1 in aaa-server group RANDOMGROUP as ACTIVE\n",
    "@version": "1",
    "ServiceName": "service-x",
    "syslog_pri": "162",
    "host": "127.0.0.14",
    "ciscotag": "ASA-2-113023",
    "@timestamp": "2020-05-12T08:20:54.108Z",
    "unique_hostname": "hostname.example.com",
    "SID": "s112731",
    "timestamp": "May 12 2020 10:20:54"
  },
  "fields": {
    "@timestamp": [
      "2020-05-12T08:20:54.108Z"
    ]
  },
  "highlight": {
    "ciscotag": [
      "@kibana-highlighted-field@ASA@/kibana-highlighted-field@-@kibana-highlighted-field@2@/kibana-highlighted-field@-@kibana-highlighted-field@113022@/kibana-highlighted-field@"
    ]
  },
  "sort": [
    1589271654108
  ]
}

Notice the field "ciscotag" which contains the value "ASA-2-113022" if there is a "down" event. If the value of the field is "ASA-2-113023" it's an "up" event.

So we basically want to create an alert if there is a down event for a host and there has not been a up event for the last 15 minutes for the same host. Is that possible with a terms aggregation?

Could you please take a look at the post i did above?

Anyone?

My suggestion from above still holds true, you can solve this with a terms agg on the ciscotag field nested within a terms agg on the unique_hostname for example.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.