# How to prevent terms aggregation from causing OOM on data nodes

**URL:** https://discuss.elastic.co/t/how-to-prevent-terms-aggregation-from-causing-oom-on-data-nodes/190941
**Category:** Elasticsearch
**Created:** [July 17, 2019, 9:44am UTC](https://discuss.elastic.co/t/how-to-prevent-terms-aggregation-from-causing-oom-on-data-nodes/190941 "2019-07-17T09:44:15Z")
**Posts on this page:** 5
**Page:** 1

<div class="post-metadata">

### Author: ![stephen.palfreyman](https://avatars.discourse-cdn.com/v4/letter/s/848f3c/32.png) [@stephen.palfreyman](https://discuss.elastic.co/u/stephen.palfreyman)
#### Post date: [July 17, 2019, 9:44am UTC](https://discuss.elastic.co/t/how-to-prevent-terms-aggregation-from-causing-oom-on-data-nodes/190941/1 "2019-07-17T09:44:15Z")

</div>

We have experienced a similar issue to that described in [this thread](https://discuss.elastic.co/t/elasticsearch-6-6-2-constantly-failing-with-out-of-memory-errors/173669) whereby one of our users tries to create a huge data table in a visualization in Kibana. This table has 2 buckets which result in 180,000 discrete values, which, perhaps unsurprisingly, cause serious memory issues in the cluster to the point some data nodes go OOM and drop out of the cluster. I already know that it isn't an appropriate usage of a data table, and we've worked around the issue by using a composite aggregation instead. However, I would like to know if there are any settings in elasticsearch which would protect elasticsearch data nodes from going OOM in this circumstance. We have hundreds of users and we cannot educate them all and ensure that they won't run crazy queries like this!  
Thanks!

---

<div class="post-metadata">

### Author: ![Mark\_Harwood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mark_harwood/32/10538_2.png) [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)
#### Post date: [July 17, 2019, 9:59am UTC](https://discuss.elastic.co/t/how-to-prevent-terms-aggregation-from-causing-oom-on-data-nodes/190941/2 "2019-07-17T09:59:14Z")

</div>

Hi Stephen,  
What version are you using?  
There's a [cluster setting](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-bucket.html) to limit the number of buckets produced by any aggregation.  
Since version 7.0 this has been defaulted to 10000 but prior to that you would have to set that manually.  
Also, version 7.0 of elasticsearch introduced the [real memory circuit breaker](https://www.elastic.co/blog/improving-node-resiliency-with-the-real-memory-circuit-breaker)

---

<div class="post-metadata">

### Author: ![stephen.palfreyman](https://avatars.discourse-cdn.com/v4/letter/s/848f3c/32.png) [@stephen.palfreyman](https://discuss.elastic.co/u/stephen.palfreyman)
#### Post date: [July 17, 2019, 10:07am UTC](https://discuss.elastic.co/t/how-to-prevent-terms-aggregation-from-causing-oom-on-data-nodes/190941/3 "2019-07-17T10:07:21Z")

</div>

Thanks Mark,  
That's really useful information - we are on 6.8 but looking to upgrade to 7.2 in the near future, I will look at implementing these as appropriate.  
Thanks,  
Steve

---

<div class="post-metadata">

### Author: ![Mark\_Harwood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mark_harwood/32/10538_2.png) [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)
#### Post date: [July 17, 2019, 10:15am UTC](https://discuss.elastic.co/t/how-to-prevent-terms-aggregation-from-causing-oom-on-data-nodes/190941/4 "2019-07-17T10:15:52Z")

</div>

Also of note is the new [dataframe](https://www.elastic.co/guide/en/elastic-stack-overview/7.2/ml-dataframes.html) functionality. Depending on what your users are doing with aggs this may be useful. Users often use aggs at query-time to join related data that is perhaps best joined at index-time for better analysis.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [August 14, 2019, 10:15am UTC](https://discuss.elastic.co/t/how-to-prevent-terms-aggregation-from-causing-oom-on-data-nodes/190941/5 "2019-08-14T10:15:53Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
