# Logs is sending into Elasticsearch takes much more disk space than in txt format

**URL:** <https://discuss.elastic.co/t/logs-is-sending-into-elasticsearch-takes-much-more-disk-space-than-in-txt-format/84717>\
**Category:** Elasticsearch\
**Created:** [May 5, 2017, 1:21pm UTC](https://discuss.elastic.co/t/logs-is-sending-into-elasticsearch-takes-much-more-disk-space-than-in-txt-format/84717 "2017-05-05T13:21:30Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![emilio](https://avatars.discourse-cdn.com/v4/letter/e/f475e1/32.png) [@emilio](https://discuss.elastic.co/u/emilio)\
**Post date:** [May 5, 2017, 1:21pm UTC](https://discuss.elastic.co/t/logs-is-sending-into-elasticsearch-takes-much-more-disk-space-than-in-txt-format/84717/1 "2017-05-05T13:21:30Z")

</div>

Dear all,

Logs is sending into Elasticsearch takes much more disk space than in txt format.

I have 20 application servers with ~ 5 apps on every host. So daily volume of logs ~ 10GB. But in ES it equals ~ 90-100GB.

How can i optimize it and reduce used disk space?

Quick response will be greatly appreciated.

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [May 5, 2017, 2:09pm UTC](https://discuss.elastic.co/t/logs-is-sending-into-elasticsearch-takes-much-more-disk-space-than-in-txt-format/84717/2 "2017-05-05T14:09:34Z")

</div>

You can first visit the mapping. It's by default generated automatically.  
May be sometimes you don't need to index some fields or you don't want to compute aggregations?  
May be you want to disable `_all` field which is enabled by default...

---

<div class="post-metadata">

**Author:** ![emilio](https://avatars.discourse-cdn.com/v4/letter/e/f475e1/32.png) [@emilio](https://discuss.elastic.co/u/emilio)\
**Post date:** [May 5, 2017, 2:23pm UTC](https://discuss.elastic.co/t/logs-is-sending-into-elasticsearch-takes-much-more-disk-space-than-in-txt-format/84717/3 "2017-05-05T14:23:16Z")

</div>

Could you please give more detailed info about? Or provide some links where i can read about.

Thanks in advance, David

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [May 5, 2017, 2:41pm UTC](https://discuss.elastic.co/t/logs-is-sending-into-elasticsearch-takes-much-more-disk-space-than-in-txt-format/84717/4 "2017-05-05T14:41:50Z")

</div>

[https://www.elastic.co/guide/en/elasticsearch/reference/5.4/mapping-all-field.html](https://www.elastic.co/guide/en/elasticsearch/reference/5.4/mapping-all-field.html)  
[https://www.elastic.co/guide/en/elasticsearch/reference/5.4/mapping.html](https://www.elastic.co/guide/en/elasticsearch/reference/5.4/mapping.html)

Might help

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [May 5, 2017, 5:56pm UTC](https://discuss.elastic.co/t/logs-is-sending-into-elasticsearch-takes-much-more-disk-space-than-in-txt-format/84717/5 "2017-05-05T17:56:19Z")

</div>

You can look at [this blog post](https://www.elastic.co/blog/elasticsearch-storage-the-true-story-2.0). It is for ES 2.x, but a lot of it is still valid. Another thing that can affect storage is how large shards you have. Compression tend to be more efficient for larger shards. What is your average shard size?

---

<div class="post-metadata">

**Author:** ![emilio](https://avatars.discourse-cdn.com/v4/letter/e/f475e1/32.png) [@emilio](https://discuss.elastic.co/u/emilio)\
**Post date:** [May 8, 2017, 7:40am UTC](https://discuss.elastic.co/t/logs-is-sending-into-elasticsearch-takes-much-more-disk-space-than-in-txt-format/84717/6 "2017-05-08T07:40:57Z")

</div>

I don't know.

How can i get know how large is my shard size? Is there any APIs?

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [May 8, 2017, 7:52am UTC](https://discuss.elastic.co/t/logs-is-sending-into-elasticsearch-takes-much-more-disk-space-than-in-txt-format/84717/7 "2017-05-08T07:52:25Z")

</div>

Use the \_cat/indices or \_cat/shards APIs.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 5, 2017, 10:35am UTC](https://discuss.elastic.co/t/logs-is-sending-into-elasticsearch-takes-much-more-disk-space-than-in-txt-format/84717/9 "2017-06-05T10:35:32Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
