Pipeline Aggregation and Counting Buckets

Elastic Stack Elasticsearch
1

I'm running ES-6.2 and have an index that contains Windows eventlog data. I'm trying to create a query using pipeline aggregations to aggregate failed logins by username and then again by the count of actual failures. I'm new to aggregations, but found this discussion helpful, but not quite what I need:

Here is a query that works (without being able to aggregate on count):

GET windows-2019.03.21/_search
{
  "query": {
    "bool": {
      "must": {
        "match_phrase": {
          "keywords": "Audit Failure"
        }
      }
    }
  },
  "aggs": {
    "by_username": {
      "terms": {
        "field": "event_data.TargetUserName"
      }
    }
  }
}

When I add in the pipeline aggregation, I get an error. Here is that query:

GET windows-2019.03.11/_search
{
  "query": {
    "bool": {
      "filter": {
        "range": {
          "@timestamp": {
            "from": "now-10m",
            "to": "now"
          }
        }
      },
      "must": {
        "match_phrase": {"keywords": "Audit Failure"}
      }
    }
  },
  "aggs": {
    "by_username": {
      "terms": {
        "field": "event_data.TargetUserName",
        "size": 10
      },
      "aggs": {
        "my_threshold": {
          "bucket_selector": {
            "buckets_path": {
              "the_doc_count": "_count"
            },
            "script": "the_doc_count >= 100"
          }
        }
      }
    }
  }
}

And here is the error:

{
  "error": {
    "root_cause": [],
    "type": "search_phase_execution_exception",
    "reason": "",
    "phase": "fetch",
    "grouped": true,
    "failed_shards": [],
    "caused_by": {
      "type": "script_exception",
      "reason": "compile error",
      "script_stack": [
        "the_doc_count >= 100",
        "^---- HERE"
      ],
      "script": "the_doc_count >= 100",
      "lang": "painless",
      "caused_by": {
        "type": "illegal_argument_exception",
        "reason": "Variable [the_doc_count] is not defined."
      }
    }
  },
  "status": 503
}

I feel like I'm really close, but just need a little help seeing my error. Thanks for anything you can offer.

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.