How to pass the aggs top 1 value to the query

Elastic Observability
1

I am trying to create an alert using ElasticDSL 7.17 query. I need to filter the documents based on certain condition and group the documents on a field and get the top first group. My query should match only the top grouped value. The below query matches only the filter criteria and i want to filter further with the top grouping name . SO i need to pass the aggs top field to the query somehow. Is it possible. Pls help.

{
  "size": 0,
  "aggregations": {
    "my_agg": {
      "filter": {
        "bool": {
          "must": [
            { "term": { "service.name": "inputservicename" }},
            { "term": { "grouping_key": "inputkey" }}
          ]
        }
      },
      "aggs": {
        "top_my_field": {
          "terms": {
            "field": "grouping_name",
            "size": 1
          }
        }
      }
    }

  },
  "query": {
    "bool": {
      "must": [
        { "match_phrase": { "service.name": "inputservicename" }},
        { "match_phrase": { "grouping_key": "inputkey" }}
      ]
    }
  }
}
2

You can do a top_hits aggregation and use the sort option to sort by whatever field makes that the "top one".

3

With the existing query, I am able to fetch the top first grouping name. I want to include this top grouping name in the query part so that the number of matched documents will be further more reduced. If i don't pass the top grouping name to the query part, the documents is just filtered by servicename and key. Instead i want to filter by service name ,key and the top value of grouping name. I am concerned on the number of matching documents because i am creating an alert if the number of matching documents > 10 ( for Ex).

4

It's not clear to me, are you using the elasticsearch query rule type? Elasticsearch query | Kibana Guide [8.7] | Elastic

You mentioned 7.x, the doc above is for 8.7. There are some additional features in 8.7, including grouping, which does a top-level agg. It may fit your needs.

Also note that in 7.x, the only top-level fields we use in the DSL are query - So aggregations should be completely ignored when the rule runs. In 8.7, we also allow fields, _source and runtime_mappings, to fine-tune the query a bit more, and control what is output for fields.

1 Like
5

Patrick is correct - if you truly require the use of aggregations, you'll need to use Watcher, not Kibana Alerts.

1 Like
6

Thank you so much! Yes that's the issue am facing, aggs are ignored while rule runs. Thanks for making it clear, that it is how 7.17 behaves.

7

Thanks for the suggestion! Will try to use watcher instead.