# Nesting high disk usage

**URL:** <https://discuss.elastic.co/t/nesting-high-disk-usage/40626>\
**Category:** Elasticsearch\
**Created:** [February 1, 2016, 10:32am UTC](https://discuss.elastic.co/t/nesting-high-disk-usage/40626 "2016-02-01T10:32:42Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![Benjamin\_Gathmann](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/benjamin_gathmann/32/7561_2.png) [@Benjamin\_Gathmann](https://discuss.elastic.co/u/Benjamin_Gathmann)\
**Post date:** [February 1, 2016, 10:32am UTC](https://discuss.elastic.co/t/nesting-high-disk-usage/40626/1 "2016-02-01T10:32:42Z")

</div>

Hi there,

I have complex documents where I need a lot of nesting on more than 100 different nodes.  
I first indexed my documents without nesting, which gave me a disk size ratio of around 1 : 0.4 (i.e. the ES index uses less than half the disk space of the original documents).  
Next, I indexed the documents with nesting added, and quite shockingly, the ratio is now around 1:2.5 (so now the index is gobbling up extreme amounts of disk space).  
Can somebody guess and explain what is going on?

---

<div class="post-metadata">

**Author:** ![Benjamin\_Gathmann](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/benjamin_gathmann/32/7561_2.png) [@Benjamin\_Gathmann](https://discuss.elastic.co/u/Benjamin_Gathmann)\
**Post date:** [February 1, 2016, 8:49pm UTC](https://discuss.elastic.co/t/nesting-high-disk-usage/40626/2 "2016-02-01T20:49:07Z")

</div>

Naturally, the suspicion is that the large number of nested objects leads to the increase in disk usage. My original 562 documents result in 10,321,880,766 (that's 10 billion, yes!) documents on my node.  
To get a better picture, it would be helfpul to know how many of each nested type I have. The [Indices Stat API](https://www.elastic.co/guide/en/elasticsearch/reference/2.x/indices-stats.html) only gives me the total number of docs.  
Is there a way to get a break-down of these numbers by nested document path?

---

<div class="post-metadata">

**Author:** ![Benjamin\_Gathmann](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/benjamin_gathmann/32/7561_2.png) [@Benjamin\_Gathmann](https://discuss.elastic.co/u/Benjamin_Gathmann)\
**Post date:** [February 2, 2016, 8:26am UTC](https://discuss.elastic.co/t/nesting-high-disk-usage/40626/3 "2016-02-02T08:26:46Z")

</div>

Sorry, the 10 billions were wrong, that was size\_in\_bytes. 😉  
The docs count is actually 14,183,108 (so still considerably high).

---

<div class="post-metadata">

**Author:** ![Benjamin\_Gathmann](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/benjamin_gathmann/32/7561_2.png) [@Benjamin\_Gathmann](https://discuss.elastic.co/u/Benjamin_Gathmann)\
**Post date:** [February 2, 2016, 10:39am UTC](https://discuss.elastic.co/t/nesting-high-disk-usage/40626/4 "2016-02-02T10:39:54Z")

</div>

Hello again,  
I have played around with setting index.codec to best\_compression, but this did not have any effect.  
Other approaches I have figured out from "The true strory behind Elasticsearch storage requirements (part 1+2):

- disable doc\_values (I have loads of not\_analyzed strings)
- set index:no for any field that I probably will not search on  
What would others recommend?

---

<div class="post-metadata">

**Author:** ![Benjamin\_Gathmann](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/benjamin_gathmann/32/7561_2.png) [@Benjamin\_Gathmann](https://discuss.elastic.co/u/Benjamin_Gathmann)\
**Post date:** [February 2, 2016, 11:47am UTC](https://discuss.elastic.co/t/nesting-high-disk-usage/40626/5 "2016-02-02T11:47:57Z")

</div>

After further testing, there is just one thing I really don't get:  
Why do nested documents use so much disk space?

Please, somebody give me a clue.

---

<div class="post-metadata">

**Author:** ![Benjamin\_Gathmann](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/benjamin_gathmann/32/7561_2.png) [@Benjamin\_Gathmann](https://discuss.elastic.co/u/Benjamin_Gathmann)\
**Post date:** [February 22, 2016, 10:42am UTC](https://discuss.elastic.co/t/nesting-high-disk-usage/40626/6 "2016-02-22T10:42:45Z")

</div>

I have received a helpful reply on Github, see here:

> <https://github.com/elastic/elasticsearch/issues/16721#issuecomment-186205531>

  
Of course, further comments and details are welcome.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 5, 2017, 11:14pm UTC](https://discuss.elastic.co/t/nesting-high-disk-usage/40626/7 "2017-07-05T23:14:29Z")

</div>


