# High cardinality multi bucket terms aggregation - huge memory consumption

**URL:** https://discuss.elastic.co/t/high-cardinality-multi-bucket-terms-aggregation-huge-memory-consumption/235530
**Category:** Elasticsearch
**Created:** [June 3, 2020, 11:13am UTC](https://discuss.elastic.co/t/high-cardinality-multi-bucket-terms-aggregation-huge-memory-consumption/235530 "2020-06-03T11:13:58Z")
**Posts on this page:** 3
**Page:** 1

<div class="post-metadata">

### Author: ![str8y](https://avatars.discourse-cdn.com/v4/letter/s/f05b48/32.png) [@str8y](https://discuss.elastic.co/u/str8y)
#### Post date: [June 3, 2020, 11:13am UTC](https://discuss.elastic.co/t/high-cardinality-multi-bucket-terms-aggregation-huge-memory-consumption/235530/1 "2020-06-03T11:13:58Z")

</div>

Hi all,  
I' m pretty new to the elastic stack but I'm facing an issue which I don't understand.  
I tried to reproduce it with the easiest example:

My documents look like this:

```auto
{
    "a1" : "3abcdefghi",
    "a2" : "3abcdefghi_a",
    "dummyA" : "dmihnibdpu"
}

```

The index contains 1M of these documents, with a1 and a2 having 10K different values and dummyA having 1M different values.

The index size is around 120MB.

The index mapping looks as follows:

```auto
{
  "mapping": {
    "_doc": {
      "dynamic_templates": [
        {
          "strings": {
            "match_mapping_type": "string",
            "mapping": {
              "fields": {
                "query": {
                  "type": "text"
                }
              },
              "type": "keyword"
            }
          }
        }
      ],
      "properties": {
        "a1": {
          "type": "keyword",
          "fields": {
            "query": {
              "type": "text"
            }
          }
        },
        "a2": {
          "type": "keyword",
          "fields": {
            "query": {
              "type": "text"
            }
          }
        },
        "dummyA": {
          "type": "keyword",
          "fields": {
            "query": {
              "type": "text"
            }
          }
        }
      }
    }
  }
}

```

Elasticsearch runs with 7GB heap.

The following query is not executable since it would need more than the 7GB heap memory (parent circuit breaker kicks in):

```auto
GET /test/_search
{
  "size": 0,
  "aggs": {
    "a1": {
      "terms": {
        "field": "a1",
        "size": 10000
      },
      "aggs": {
        "a2": {
          "terms": {
            "field": "a2",
            "size": 10000
          }
        }
      }
    }
  }
}

```

If I use execution\_hint: map like so:

```auto
GET /test/_search
{
  "size": 0,
  "aggs": {
    "a1": {
      "terms": {
        "field": "a1",
        "size": 10000,
        "execution_hint": "map"
      },
      "aggs": {
        "a2": {
          "terms": {
            "field": "a2",
            "size": 10000,
            "execution_hint": "map"
          }
        }
      }
    }
  }
}

```

the query returns in around 1s with the result.

As far as I understood, execution\_hint: map should actually lead to larger requests and hence consume more memory?

The size of the requests is indeed larger but the overall memory consumption with execution\_hint map is much lower than the memory consumption with execution\_hint global\_ordinals, which is from my understanding the default.

Does anyone know and can explain this huge memory consumption in the default mode for an index that has only 120MB?

Any help would be very cool.

Thanks 🙂

---

<div class="post-metadata">

### Author: ![tobiasf](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tobiasf/32/45910_2.png) [@tobiasf](https://discuss.elastic.co/u/tobiasf)
#### Post date: [June 15, 2020, 8:44am UTC](https://discuss.elastic.co/t/high-cardinality-multi-bucket-terms-aggregation-huge-memory-consumption/235530/3 "2020-06-15T08:44:45Z")

</div>

I have the same problem and also cannot explain it.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 13, 2020, 8:44am UTC](https://discuss.elastic.co/t/high-cardinality-multi-bucket-terms-aggregation-huge-memory-consumption/235530/4 "2020-07-13T08:44:59Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.