# Terms aggregation very slow - but very fast with murmur3

**URL:** <https://discuss.elastic.co/t/terms-aggregation-very-slow-but-very-fast-with-murmur3/103760>\
**Category:** Elasticsearch\
**Created:** [October 12, 2017, 3:53pm UTC](https://discuss.elastic.co/t/terms-aggregation-very-slow-but-very-fast-with-murmur3/103760 "2017-10-12T15:53:36Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![ebuildy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ebuildy/32/6070_2.png) [@ebuildy](https://discuss.elastic.co/u/ebuildy)\
**Post date:** [October 12, 2017, 3:53pm UTC](https://discuss.elastic.co/t/terms-aggregation-very-slow-but-very-fast-with-murmur3/103760/1 "2017-10-12T15:53:36Z")

</div>

I do a nested aggregation like this:

```
"aggregations": {
    "query": {
      "terms": {
        "field": "query",
        "size": 1000
      },
      "aggs": {
        "url": {
          "terms": {
            "field": "url.keyword",
            "size": 10
          }
        }
      }
    }
  }

```

Both query and url.keyword are stored as keyword. The query takes 6 secondes with ES 5. But 30 ms if I use murmur3 plugin. Hence my question, why elasticsearch terms aggregation dont use a hash for the "group by" then lookup to get the real field value?

 ![5b059367-145a-4bf9-aec1-7eb0c1358dfa](https://us1.discourse-cdn.com/elastic/original/3X/7/6/76fde99abaa112ca8924ab9f46a73fbb7c27d800.jpg) ![4368213b-dae5-4b59-b7df-d9ca1109bb7f](https://us1.discourse-cdn.com/elastic/original/3X/2/1/21b999eb4216648c433b180a058b04bb11cca42b.jpg)

---

<div class="post-metadata">

**Author:** ![Mark\_Harwood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mark_harwood/32/10538_2.png) [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)\
**Post date:** [October 12, 2017, 5:29pm UTC](https://discuss.elastic.co/t/terms-aggregation-very-slow-but-very-fast-with-murmur3/103760/2 "2017-10-12T17:29:19Z")

</div>

> [@ebuildy](#):
>
> why elasticsearch terms aggregation dont use a hash for the "group by" then lookup to get the real field value?

We don't want to take responsibility for any hash collision inaccuracies - that can be a decision you take in your app code e.g. you could look at using breadth\_first collect mode on your hashed term then embed a full version of the term as a child agg.

---

<div class="post-metadata">

**Author:** ![ebuildy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ebuildy/32/6070_2.png) [@ebuildy](https://discuss.elastic.co/u/ebuildy)\
**Post date:** [October 13, 2017, 3:19pm UTC](https://discuss.elastic.co/t/terms-aggregation-very-slow-but-very-fast-with-murmur3/103760/3 "2017-10-13T15:19:08Z")

</div>

Ho I see,

top\_hits is working fine ! I got results in 3 secondes, which is very slow compared to 300ms without it ..... For now, I got exactly the same results, murmur3 is very fast.

So, murmur3 team should implement a kind of "terms\_hash" aggregation 😉

There is already an approximate aggregation : cardinality where we can configure the precision, why you dont do the same with terms?

---

<div class="post-metadata">

**Author:** ![Mark\_Harwood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mark_harwood/32/10538_2.png) [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)\
**Post date:** [October 13, 2017, 4:11pm UTC](https://discuss.elastic.co/t/terms-aggregation-very-slow-but-very-fast-with-murmur3/103760/4 "2017-10-13T16:11:42Z")

</div>

Where possible, internally we prefer to count using ordinals instead of strings. Ordinals are numbers used as a placeholder for a unique string.  
Normally ordinals are used if the terms agg is the root of the aggs tree but we will flip to a different policy for deeper nested sections of the agg tree relying on maps of strings.

**[WARNING - EXPERT LEVEL FEATURE THAT MAY WREAK HAVOC......]**  
In your dev environment - you can look at setting a choice of execution\_hint [1] to influence the strategy used for collecting buckets. The `global_ordinals` will likely be hugely expensive, allocating a large array per parent agg bucket and the health of your nodes may be an issue. However `global_ordinals_hash` may be an option to look at. _ **Take care with these settings and only try first in a non-production environment.** _

[1] [https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-bucket-terms-aggregation.html#search-aggregations-bucket-terms-aggregation-execution-hint](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-bucket-terms-aggregation.html#search-aggregations-bucket-terms-aggregation-execution-hint)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 10, 2017, 4:11pm UTC](https://discuss.elastic.co/t/terms-aggregation-very-slow-but-very-fast-with-murmur3/103760/5 "2017-11-10T16:11:46Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.