# How to check total required heap for ES 8.8 **URL:** \ **Category:** Elasticsearch\ **Created:** [June 24, 2023, 10:45pm UTC](https://discuss.elastic.co/t/how-to-check-total-required-heap-for-es-8-8/336821 "2023-06-24T22:45:04Z")\ **Posts on this page:** 15\ **Page:** 1
**Author:** ![Geunmoon\_Oh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/geunmoon_oh/32/7507_2.png) [@Geunmoon\_Oh](https://discuss.elastic.co/u/Geunmoon_Oh)\ **Post date:** [June 24, 2023, 10:45pm UTC](https://discuss.elastic.co/t/how-to-check-total-required-heap-for-es-8-8/336821/1 "2023-06-24T22:45:04Z")
To set my ES node's heap minimum, I referenced Size your shards ([Size your shards | Elasticsearch Guide [8.8] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/size-your-shards.html#each-shard-has-overhead)). "total\_deduplicated\_mapping\_size" : "4.1kb“ "total\_estimated\_overhead" : “13.5mb“ "extra heap for other overheads": 0.5GB "master node" : 1GB Finally I set '-Xms2g -Xmx2g' in jvm.options. then I restarted ES. But, this ES Service go down due to OOM. I think fielddata cache should be considerated. How i can set Heap Memory size minimum without go down ? ---
**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\ **Post date:** [June 25, 2023, 5:34am UTC](https://discuss.elastic.co/t/how-to-check-total-required-heap-for-es-8-8/336821/2 "2023-06-25T05:34:07Z")
From [these docs](https://www.elastic.co/guide/en/elasticsearch/reference/current/size-your-shards.html#_consider_additional_heap_overheads): > 0.5GB of extra heap will suffice for many reasonable workloads, and you may need even less if your workload is very light while heavy workloads may require more. It's unfortunate that we cannot be more precise here, but it depends on so many other factors related to your workload. In your case, it sounds like you need to allow more than 0.5GB for your workload. > [@Geunmoon\_Oh](#): > > I think fielddata cache should be considerated. Why do you think the fielddata cache is the problem here? ---
**Author:** ![Geunmoon\_Oh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/geunmoon_oh/32/7507_2.png) [@Geunmoon\_Oh](https://discuss.elastic.co/u/Geunmoon_Oh)\ **Post date:** [June 25, 2023, 8:32am UTC](https://discuss.elastic.co/t/how-to-check-total-required-heap-for-es-8-8/336821/3 "2023-06-25T08:32:29Z")
I tested heap with fielddata cache. 1. ES Service was loaded with heap 2g setting. but this was stopped due to OOM. I felt 2g is not enough. so I added more heap memory. 2. ES Service was loaded with heap 3g setting. but this was also stopped due to OOM. I felt heap 3g is not enough too. To check detail, I captured heap dump from ES. and I found fielddata do something wrong. ![heapdump](https://us1.discourse-cdn.com/elastic/original/3X/1/f/1f6cb6b0678744f2e64aa34680cdf2657e099c83.png) 1. So I restricted fielddata size. ES Service was loaded with heap 3g, fielddata.cache.size:1GB setting. the service was go well. ![heap31g_fielddata.cache.size1GB](https://us1.discourse-cdn.com/elastic/original/3X/2/1/21cef116c56bab6a97d61641c11f86dac9a88d8f.png) Finaly I realized "total\_deduplicated\_mapping\_size","total\_estimated\_overhead","extra heap for other overheads" and "fielddata cache size" are considerated for heap size. ---
**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\ **Post date:** [June 25, 2023, 9:39am UTC](https://discuss.elastic.co/t/how-to-check-total-required-heap-for-es-8-8/336821/4 "2023-06-25T09:39:40Z")
Thanks, that seems like a compelling analysis. However I'm puzzled because the [fielddata circuit breaker](https://www.elastic.co/guide/en/elasticsearch/reference/current/circuit-breaker.html#fielddata-circuit-breaker) should prevent this, limiting the size of this cache to 40% of the heap by default. Do you know why it didn't? For instance, what does `GET /_nodes/_all/stats/breaker?filter_path=nodes.*.breakers.fielddata` report? ---
**Author:** ![Geunmoon\_Oh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/geunmoon_oh/32/7507_2.png) [@Geunmoon\_Oh](https://discuss.elastic.co/u/Geunmoon_Oh)\ **Post date:** [June 25, 2023, 10:19am UTC](https://discuss.elastic.co/t/how-to-check-total-required-heap-for-es-8-8/336821/5 "2023-06-25T10:19:06Z")
I checked it with my last environments (ES Service with heap 3g, fielddata.cache.size:1GB setting). "GET /\_nodes/\_all/stats/breaker?filter\_path=nodes.\*.breakers.fielddata" returned below ``` { "nodes": { "K6V95L0pR36L-_99LIapdw": { "breakers": { "fielddata": { "limit_size_in_bytes": 1288490188, "limit_size": "1.1gb", "estimated_size_in_bytes": 1047111152, "estimated_size": "998.6mb", "overhead": 1.03, "tripped": 0 } } } } } ``` PS. I used ES version 8.6 (not V8.8) ---
**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\ **Post date:** [June 25, 2023, 10:25am UTC](https://discuss.elastic.co/t/how-to-check-total-required-heap-for-es-8-8/336821/6 "2023-06-25T10:25:34Z")
That makes sense, but what about in the case where you don't set `fielddata.cache.size`? The `"limit_size": "1.1gb"` should still apply. Alternatively, the heap dump you captured showed 2.5GiB of heap being retained by the cache. Is that reflected in the circuit breaker and/or the stats (which you can compute from all the `org.elasticsearch.index.fielddata.ShardFieldData#perFieldTotals` maps)? Or is Elasticsearch not tracking some of this memory usage? ---
**Author:** ![Geunmoon\_Oh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/geunmoon_oh/32/7507_2.png) [@Geunmoon\_Oh](https://discuss.elastic.co/u/Geunmoon_Oh)\ **Post date:** [June 26, 2023, 3:30am UTC](https://discuss.elastic.co/t/how-to-check-total-required-heap-for-es-8-8/336821/7 "2023-06-26T03:30:09Z")
i retested ES Service with heap 3g setting (no setting `fielddata.cache.size`). ES returned about fielddata below before stopping. Maybe Fielddata circuit breaker didn't control it. ``` { "nodes": { "K6V95L0pR36L-_99LIapdw": { "breakers": { "fielddata": { "limit_size_in_bytes": 1288490188, "limit_size": "1.1gb", "estimated_size_in_bytes": 2766456144, "estimated_size": "2.5gb", "overhead": 1.03, "tripped": 0 } } } } } ``` ---
**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\ **Post date:** [June 26, 2023, 7:12am UTC](https://discuss.elastic.co/t/how-to-check-total-required-heap-for-es-8-8/336821/8 "2023-06-26T07:12:27Z")
Thanks, that's helpful. I think this is a bug so I opened an issue in Github for you: > > > A \[user on the forums\](https://discuss.elastic.co/t/how-to-check-total-required-…heap-for-es-8-8/336821/7?u=davidturner) reported experiencing OOMEs and when they analysed the heap dump they found that a high fraction of their 3GiB heap was used by the fielddata cache. \`GET /\_nodes/\_all/stats/breaker?filter\_path=nodes.\*.breakers.fielddata\` agrees: > > \`\`\` > { > "nodes": { > "K6V95L0pR36L-\_99LIapdw": { > "breakers": { > "fielddata": { > "limit\_size\_in\_bytes": 1288490188, > "limit\_size": "1.1gb", > "estimated\_size\_in\_bytes": 2766456144, > "estimated\_size": "2.5gb", > "overhead": 1.03, > "tripped": 0 > } > } > } > } > } > \`\`\` > > They have worked around this problem by setting \`indices.fielddata.cache.size: 1gb\` but I think it's a bug for the fielddata cache to grow without bounds like this. ---
**Author:** ![Geunmoon\_Oh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/geunmoon_oh/32/7507_2.png) [@Geunmoon\_Oh](https://discuss.elastic.co/u/Geunmoon_Oh)\ **Post date:** [June 26, 2023, 1:19pm UTC](https://discuss.elastic.co/t/how-to-check-total-required-heap-for-es-8-8/336821/9 "2023-06-26T13:19:37Z")
Thank you about fixing bug. but I need more. As i said above, ES Service was loaded with heap 3g, fielddata.cache.size:1GB setting. And when i add more index, heap is more required.. but, "total\_deduplicated\_mapping\_size", "total\_estimated\_overhead" values are uppered little bit. to check sufficient heap size, what i have to check more? ---
**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\ **Post date:** [June 26, 2023, 1:48pm UTC](https://discuss.elastic.co/t/how-to-check-total-required-heap-for-es-8-8/336821/10 "2023-06-26T13:48:17Z")
I don't have a good answer to this (at least nothing more specific than "your workload"). If you limit the fielddata cache to 1GiB, what else is consuming too much heap in your system? ---
**Author:** ![Geunmoon\_Oh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/geunmoon_oh/32/7507_2.png) [@Geunmoon\_Oh](https://discuss.elastic.co/u/Geunmoon_Oh)\ **Post date:** [June 27, 2023, 9:06am UTC](https://discuss.elastic.co/t/how-to-check-total-required-heap-for-es-8-8/336821/11 "2023-06-27T09:06:02Z")
I also use ES 5.6. the total number of org.elasticsearch.index.IndexService objects is equal to the number of open indexes In ES 5.6. But the total number of org.elasticsearch.index.IndexService objects is equal to the number of all indexes In ES 8.6. why so many indexserves objects are loaded in the Service? Can i reduce the count for small heap size? ---
**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\ **Post date:** [June 27, 2023, 9:36am UTC](https://discuss.elastic.co/t/how-to-check-total-required-heap-for-es-8-8/336821/12 "2023-06-27T09:36:43Z")
> Can i reduce the count for small heap size? No. The behaviour in 5.6 was a bug, long-since fixed. ---
**Author:** ![Geunmoon\_Oh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/geunmoon_oh/32/7507_2.png) [@Geunmoon\_Oh](https://discuss.elastic.co/u/Geunmoon_Oh)\ **Post date:** [June 27, 2023, 9:31pm UTC](https://discuss.elastic.co/t/how-to-check-total-required-heap-for-es-8-8/336821/13 "2023-06-27T21:31:12Z")
In ES 5.6, if 10 indexes exist and 2 indexes open, 2 org.elasticsearch.index.IndexService objects loaded. but, In ES 8.6, although 10 indexes exist and 2 indexes open, 10 org.elasticsearch.index.IndexService objects loaded. if 10 indexes exist and 2 indexes open, can I load only 2 org.elasticsearch.index.IndexService objects In ES 8.6? ---
**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\ **Post date:** [June 28, 2023, 2:46am UTC](https://discuss.elastic.co/t/how-to-check-total-required-heap-for-es-8-8/336821/14 "2023-06-28T02:46:06Z")
I'm not sure how this differs from your previous question. The behaviour in 5.6 that you describe was essentially due to a bug. There should be one `IndexService` for every index, open or closed, although the closed ones will be quite lightweight and won't load any field data. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 26, 2023, 2:46am UTC](https://discuss.elastic.co/t/how-to-check-total-required-heap-for-es-8-8/336821/15 "2023-07-26T02:46:15Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.