Help with aggregation performance improvement for nested aggregations

Elastic Stack Elasticsearch
1

I'm trying to find out what the underlying problem for some of my searches' and aggregations' performance is but I can't seem to find any documentation on what the terms in the profiling breakdown mean and what I can do to improve performance. Can you point me somewhere where I can find out what the following things mean (and maybe how to improve on them):

For search:

  • score
  • build_scorer_count
  • match_count
  • create_weight
  • next_doc
  • match
  • create_weight_count
  • next_doc_count
  • score_count
  • build_scorer
  • advance
  • advance_count

For aggregations:

  • reduce
  • build_aggregation
  • build_aggregation_count
  • initialize
  • initialize_count
  • reduce_count
  • collect
  • collect_count

[Update]
Aaaaaand of course, now I find https://www.elastic.co/guide/en/elasticsearch/reference/current/search-profile.html :man_facepalming:

Ok, now I can at least be more specific because it doesn't really help me that much for my specific problem :slight_smile:

Any idea why the collect phase of this aggregator takes so long? Also, is this CPU/Memory/IO bound?

{
  "type": "NestedAggregator",
  "description": "customVariables",
  "time_in_nanos": 10816958154,
  "breakdown": {
    "reduce": 0,
    "build_aggregation": 36583,
    "build_aggregation_count": 1,
    "initialize": 15959,
    "initialize_count": 1,
    "reduce_count": 0,
    "collect": 10816896702,
    "collect_count": 8908
  },
  "children": [
    {
      "type": "GlobalOrdinalsStringTermsAggregator",
      "description": "context",
      "time_in_nanos": 10410704736,
      "breakdown": {
        "reduce": 0,
        "build_aggregation": 33198,
        "build_aggregation_count": 1,
        "initialize": 2215,
        "initialize_count": 1,
        "reduce_count": 0,
        "collect": 10410563544,
        "collect_count": 105777
      }
    }
  ]
}

With this query:

{
  "profile": true,
  "size": 0,
  "aggregations": {
    "customVariables": {
      "nested": {
        "path": "customer_input.context"
      },
      "aggregations": {
        "context": {
          "terms": {
            "field": "customer_input.context.key",
            "size": 100
          }
        }
      }
    }
  }
}

And this mapping:

{
  "properties": {
    "customer_input": {
      "dynamic": "strict",
      "properties": {
        "context": {
          "type": "nested",
          "dynamic": "strict",
          "properties": {
            "key": {
              "type": "keyword",
              "eager_global_ordinals": true,
              "ignore_above": 200
            }
          }
        }
      }
    }
  }
}

It's usually just a single shard that is slow.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.