# Out Of Memory error on cardinality aggregation

**URL:** <https://discuss.elastic.co/t/out-of-memory-error-on-cardinality-aggregation/20752>\
**Category:** Elasticsearch\
**Created:** [November 14, 2014, 3:50pm UTC](https://discuss.elastic.co/t/out-of-memory-error-on-cardinality-aggregation/20752 "2014-11-14T15:50:17Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Mikalai](https://avatars.discourse-cdn.com/v4/letter/m/a587f6/32.png) [@Mikalai](https://discuss.elastic.co/u/Mikalai)\
**Post date:** [November 14, 2014, 3:50pm UTC](https://discuss.elastic.co/t/out-of-memory-error-on-cardinality-aggregation/20752/1 "2014-11-14T15:50:17Z")

</div>

Hi,

We are using terms aggregation on high cardinality field and limiting  
results to 5000 (using “size” parameter). We also have a cardinality sub  
aggregation on this terms aggregation to get the number of unique values on  
a separate field for each term returned. Such combination of aggregations  
requires a lot of memory and we are getting Out Of Memory error.

We tried this new "collect\_mode" option with "breadth\_first" setting but  
without success. Memory consumption is the same and OOM is still there.

We identified that almost all memory consumed by ByteArray object in  
HyperLogLogPlusPlus class. This object is created in HyperLogLogPlusPlus  
constructor and initialized with “initialBucketCount \<\< p” value as size  
(where initialBucketCount is estimated buckets count passed from terms  
aggregation, p is precision). We believe that with "breadth\_first" setting  
initial bucket count should be limited to 5000 (the value we use to limit  
terms aggregation results). But what we see is that initial bucket count is  
much greater than 5000 and it’s the same as without "breadth\_first" setting  
(235000 in our case).

Is it correct behavior for cardinality sub aggregation? Is there any way to  
run this set of aggregation without OOM?

Thanks in advance,

Mikalai

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/ff428e99-5ace-484f-97c6-b7dfa417799f%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/ff428e99-5ace-484f-97c6-b7dfa417799f%40googlegroups.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![AJ\_2](https://avatars.discourse-cdn.com/v4/letter/a/ac8455/32.png) [@AJ\_2](https://discuss.elastic.co/u/AJ_2)\
**Post date:** [September 29, 2015, 11:30am UTC](https://discuss.elastic.co/t/out-of-memory-error-on-cardinality-aggregation/20752/2 "2015-09-29T11:30:53Z")

</div>

For others who might bump in this situation:  
we solved our similar issue by setting [doc\_values =\> true](https://www.elastic.co/guide/en/elasticsearch/guide/current/doc-values.html) on the field with high cardinalily.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 5, 2017, 11:47pm UTC](https://discuss.elastic.co/t/out-of-memory-error-on-cardinality-aggregation/20752/3 "2017-07-05T23:47:36Z")

</div>