Only create an alert when the query gets three hits from the same device

Elastic Stack Elasticsearch
1

Hi,

We want to create an alert when there has been over three failed logon events. Our query looks like this:

POST /logstash-test-2019-11/_search
{
  "query": {
        "bool": {
          "must": [
            {
              "query_string": {
                "query": "message: \"authentication failure\" AND \"sshd\""
              }
            }
          ],
          "filter": {
            "range": {
              "@timestamp": {
                "gte": "now-30m"
              }
            }
          }
        }
      }
}

And our whole watch:

{
  "trigger": {
    "schedule": {
      "interval": "10m"
    }
  },
  "input": {
    "search": {
      "request": {
        "search_type": "query_then_fetch",
        "indices": [
          "logstash-test-*"
        ],
        "types": [],
        "body": {
          "query": {
            "bool": {
              "must": [
                {
                  "query_string": {
                    "query": "message: \"authentication failure\" AND \"sshd\""
                  }
                }
              ],
              "filter": {
                "range": {
                  "@timestamp": {
                    "gte": "now-30m"
                  }
                }
              }
            }
          }
        }
      }
    }
  },
  "condition": {
    "compare": {
      "ctx.payload.hits.total": {
        "gte": 3
      }
    }
  },
  "actions": {
    "send_email": {
      "email": {
        "profile": "standard",
        "to": [
          "user@example.com"
        ],
        "subject": "test-subject",
        "body": {
          "text": "test-text"
        }
      }
    }
  }
}

This successfully alerts whenever we get three hits of the query in last 30 minutes. However, it also creates an alert if there are three separate failed logons for separate devices. We want to make it so that it should only alert if there are at least three hits of the same device. We save the FQDN of each device in a field called FQDN, so could we somehow use that?

Thanks

2

The query needs to be refined before being able to write a proper watch. What is missing is a terms aggregation on the device. With this you will get an aggregation response, that contains the number of failed logins per device. You can also use the min_doc_count parameter to make sure only buckets with three login failures are returned. On top of that you will also need to change the condition. You might want to use a script condition that checks the size of the aggregation buckets being returned greater than 0 instead of the total hits (as those can be spread across devices).

Hope this helps!

3

Hi,

Thank you. We're almost there but need some help on the condition part.

We have the following query now:

POST /logstash-test-2019-11/_search
{
  "query": {
        "bool": {
          "must": [
            {
              "query_string": {
                "query": "message: \"authentication failure\" AND \"sshd\""
              }
            }
          ],
          "filter": {
            "range": {
              "@timestamp": {
                "gte": "now-3h"
              }
            }
          }
        }
      },
      "aggs": {
        "device": {
          "terms": {
            "field": "device.keyword",
            "min_doc_count": 3
          }
        }
      }
}

Which returns the following (some output omitted):

"aggregations" : {
    "device" : {
      "doc_count_error_upper_bound" : 0,
      "sum_other_doc_count" : 0,
      "buckets" : [
        {
          "key" : "smtpout01,
          "doc_count" : 3
        },
        {
          "key" : "smtp01",
          "doc_count" : 3
        }
      ]
    }
  }

How would i use something like bucket.key and bucket.doc_count? I want the condition to turn true if 1 or more device aggregation returns more than or equal to 3 hits.

Thank you!

4

try checking for ctx.payload.aggregations.device.buckets.size() > 0 in a script condition

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.