# Tune on Disk Usage with \_source, doc\_values and store?

**URL:** <https://discuss.elastic.co/t/tune-on-disk-usage-with-source-doc-values-and-store/156383>\
**Category:** Elasticsearch\
**Created:** [November 13, 2018, 4:10am UTC](https://discuss.elastic.co/t/tune-on-disk-usage-with-source-doc-values-and-store/156383 "2018-11-13T04:10:03Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![dongfeng](https://avatars.discourse-cdn.com/v4/letter/d/f05b48/32.png) [@dongfeng](https://discuss.elastic.co/u/dongfeng)\
**Post date:** [November 13, 2018, 4:10am UTC](https://discuss.elastic.co/t/tune-on-disk-usage-with-source-doc-values-and-store/156383/1 "2018-11-13T04:10:03Z")

</div>

I recently do some working with tune on disk usage following the doc: [https://www.elastic.co/guide/en/elasticsearch/reference/6.4/tune-for-disk-usage.html](https://www.elastic.co/guide/en/elasticsearch/reference/6.4/tune-for-disk-usage.html).

Then I tried store those values in those three settings:

1. with \_source=true (doc\_values=false, store = false)
2. with doc\_values=true (\_source=false, store = false)
3. with store=true(\_source=false, doc\_values=false)

With my Testing, I found there's nearly no difference on disk usage with those three settings when the doc values are difference (disk usage got following the doc: [https://www.elastic.co/guide/en/elasticsearch/reference/6.4/indices-stats.html](https://www.elastic.co/guide/en/elasticsearch/reference/6.4/indices-stats.html))

But in my opinion, doc\_values which stores in column-oriented fashion should use less disk spaces. So are there anything wrong?

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [November 13, 2018, 7:37am UTC](https://discuss.elastic.co/t/tune-on-disk-usage-with-source-doc-values-and-store/156383/2 "2018-11-13T07:37:40Z")

</div>

How much data are you testing with? What is your use case? What is it you are trying to achieve?

---

<div class="post-metadata">

**Author:** ![dongfeng](https://avatars.discourse-cdn.com/v4/letter/d/f05b48/32.png) [@dongfeng](https://discuss.elastic.co/u/dongfeng)\
**Post date:** [November 13, 2018, 8:18am UTC](https://discuss.elastic.co/t/tune-on-disk-usage-with-source-doc-values-and-store/156383/3 "2018-11-13T08:18:27Z")

</div>

my main purpose is to reduce disk usages. So I tried three different kinds of mapping settings. The quantity of Testing data is about 100K.

here are three types of my mapping settings:

1. enable source, disable doc\_values and store  
{  
"settings" : {  
"index.codec" : "best\_compression"  
},  
"mappings" : {  
"\_source": {  
"enabled": true  
},  
"properties" : {  
"path": {  
"type": "keyword",  
"index": false,  
"doc\_values": false,  
"store": false  
}  
}  
}  
}  
}

2. enable doc\_values, disable source and store  
{  
"settings" : {  
"index.codec" : "best\_compression"  
},  
"mappings" : {  
"\_source": {  
"enabled": false  
},  
"properties" : {  
"path": {  
"type": "keyword",  
"index": false,  
"doc\_values": true,  
"store": false  
}  
}  
}  
}  
}

3. enable store , disable source and doc\_values  
{  
"settings" : {  
"index.codec" : "best\_compression"  
},  
"mappings" : {  
"\_source": {  
"enabled": false  
},  
"properties" : {  
"path": {  
"type": "keyword",  
"index": false,  
"doc\_values": false,  
"store": true  
}  
}  
}  
}  
}

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [November 13, 2018, 8:28am UTC](https://discuss.elastic.co/t/tune-on-disk-usage-with-source-doc-values-and-store/156383/4 "2018-11-13T08:28:22Z")

</div>

100k documents does not sound like a lot of data. I would recommend using a larger data set and make sure that you also index into a single shard as compression efficiency generally improves with shard size. Also force merge down to a single segment once you are done to get a fair comparison.

This old [blog post](https://www.elastic.co/blog/elasticsearch-storage-the-true-story-2.0) shows how we did a similar comparison for a much older version of Elasticsearch.

It is also worth noting that altering the settings you described will have an impact on how you can query and potentially also reprocess your data, so make sure any side effects of mapping changes are acceptable to your use case.

---

<div class="post-metadata">

**Author:** ![dongfeng](https://avatars.discourse-cdn.com/v4/letter/d/f05b48/32.png) [@dongfeng](https://discuss.elastic.co/u/dongfeng)\
**Post date:** [November 13, 2018, 11:42am UTC](https://discuss.elastic.co/t/tune-on-disk-usage-with-source-doc-values-and-store/156383/5 "2018-11-13T11:42:15Z")

</div>

thanks a lot，I'll try to use much more docs for testing

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 11, 2018, 11:42am UTC](https://discuss.elastic.co/t/tune-on-disk-usage-with-source-doc-values-and-store/156383/6 "2018-12-11T11:42:15Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.