# 8.3 Memory calculations and nested fields **URL:** \ **Category:** Elasticsearch\ **Created:** [November 2, 2022, 1:23pm UTC](https://discuss.elastic.co/t/8-3-memory-calculations-and-nested-fields/317988 "2022-11-02T13:23:28Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![ewolfman](https://avatars.discourse-cdn.com/v4/letter/e/5f8ce5/32.png) [@ewolfman](https://discuss.elastic.co/u/ewolfman)\ **Post date:** [November 2, 2022, 1:23pm UTC](https://discuss.elastic.co/t/8-3-memory-calculations-and-nested-fields/317988/1 "2022-11-02T13:23:28Z")
Hi, Trying to better understanding the new memory planning for 8.3. > **[Size your shards | Elasticsearch Guide \[8.3\] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/8.3/size-your-shards.html#field-count-recommendation)** Do nested fields count just like other fields? My indices have nested fields. So a single index including its nested fields has 100 fields. According to the above documentation, if I have 1000 indices this means 1000 (indices) x 100 (fields) x1kb=0.1GB heap (is this correct?) 1. Do nested fields count just like other fields in the index? 2. Just to be on the safe side: as of 8.3 the number of shards is no longer relevant for memory calculations? 3. Is there no importance to the amount of data itself within the index? In other words, if there are 2 indices with the exact mapping (e.g. rollover), and one index has 100M records and the other index just has 1 single record - there is no change in memory calculations? ---
**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\ **Post date:** [November 2, 2022, 2:02pm UTC](https://discuss.elastic.co/t/8-3-memory-calculations-and-nested-fields/317988/2 "2022-11-02T14:02:38Z")
I believe nested fields count like normal fields, but the simplest answer is to upgrade to 8.5 which reports the overhead size in the node stats: > **[Size your shards | Elasticsearch Guide \[8.5\] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/8.5/size-your-shards.html#field-count-recommendation)** > [@ewolfman](#): > > the number of shards is no longer relevant for memory calculations? There's still some amount of per-shard overhead but in most setups it's not worth considering. > [@ewolfman](#): > > Is there no importance to the amount of data itself within the index? Likewise, there's still some amount of per-segment overhead but it rarely matters. ---
**Author:** ![ewolfman](https://avatars.discourse-cdn.com/v4/letter/e/5f8ce5/32.png) [@ewolfman](https://discuss.elastic.co/u/ewolfman)\ **Post date:** [November 2, 2022, 2:26pm UTC](https://discuss.elastic.co/t/8-3-memory-calculations-and-nested-fields/317988/3 "2022-11-02T14:26:02Z")
Thanks David. I can see now that the documentation section ["Data nodes should have at least 1kB of heap per field per index, plus overheads"](https://www.elastic.co/guide/en/elasticsearch/reference/8.3/size-your-shards.html#field-count-recommendation) from 8.3 & 8.4 has been replaced with ["Allow enough heap for field mappers and overheads"](https://www.elastic.co/guide/en/elasticsearch/reference/8.5/size-your-shards.html#field-count-recommendation). What I am trying to do is to plan for upgrading from 7.x to latest 8.5. So I cannot run these APIs to get answers. This is problematic as I am trying to plan ahead the memory requirements of the cluster in the long run. Planning by number of expected indices, fields etc. is possible. But since this is now dropped from the documentation it is problematic to rely on what the API returns if you don't have 8.5 already set up (sorry if I am missing something). Some questions: 1. If I use the same mapping on some **test** 8.5 cluster would the _total\_deduplicated\_mapping\_size_ be identical after I upgrade the production to 8.5? i.e. is this totally mapping related? 2. For the node stats this is more problematic as it is per node. How can I get the expected _total\_estimated\_overhead_ before I upgrade to 8.5? Does this number relate in anyway to the number of indices? I reckon that the more indices the larger this overhead becomes - is this correct? Thanks. ---
**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\ **Post date:** [November 2, 2022, 2:52pm UTC](https://discuss.elastic.co/t/8-3-memory-calculations-and-nested-fields/317988/4 "2022-11-02T14:52:44Z")
If you have enough memory in 7.x then you will be fine in 8.x. The guidance has been updated in 8.x versions because of some significant reductions in heap usage in recent versions. I would suggest doing the upgrade first without changing the size of your cluster, and once the upgrade is complete you can start to measure things and think about reducing your cluster size. ---
**Author:** ![ewolfman](https://avatars.discourse-cdn.com/v4/letter/e/5f8ce5/32.png) [@ewolfman](https://discuss.elastic.co/u/ewolfman)\ **Post date:** [November 2, 2022, 2:58pm UTC](https://discuss.elastic.co/t/8-3-memory-calculations-and-nested-fields/317988/5 "2022-11-02T14:58:55Z")
Nevertheless, is the 8.3/8.4 documentation re. the number of indices and memory calcluations still relevant and correct in 8.5? ---
**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\ **Post date:** [November 2, 2022, 3:02pm UTC](https://discuss.elastic.co/t/8-3-memory-calculations-and-nested-fields/317988/6 "2022-11-02T15:02:26Z")
The 8.3/8.4 docs do not take account of mapping deduplication, so the memory usage in that area in 8.5 should be no worse (and will often be better). Likewise, 8.5 still allows 1kiB per mapped field (see [these docs](https://www.elastic.co/guide/en/elasticsearch/reference/8.5/cluster-nodes-stats.html)) but this may well improve in future versions. ---
**Author:** ![ewolfman](https://avatars.discourse-cdn.com/v4/letter/e/5f8ce5/32.png) [@ewolfman](https://discuss.elastic.co/u/ewolfman)\ **Post date:** [November 2, 2022, 4:10pm UTC](https://discuss.elastic.co/t/8-3-memory-calculations-and-nested-fields/317988/7 "2022-11-02T16:10:04Z")
Thanks! ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [November 30, 2022, 4:11pm UTC](https://discuss.elastic.co/t/8-3-memory-calculations-and-nested-fields/317988/8 "2022-11-30T16:11:02Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.