# Recomendations on Managing Storage Size

**URL:** https://discuss.elastic.co/t/recomendations-on-managing-storage-size/232905
**Category:** Elasticsearch
**Created:** [May 16, 2020, 6:27am UTC](https://discuss.elastic.co/t/recomendations-on-managing-storage-size/232905 "2020-05-16T06:27:36Z")
**Posts on this page:** 7
**Page:** 1

<div class="post-metadata">

### Author: ![rukmalf](https://avatars.discourse-cdn.com/v4/letter/r/43a26b/32.png) [@rukmalf](https://discuss.elastic.co/u/rukmalf)
#### Post date: [May 16, 2020, 6:27am UTC](https://discuss.elastic.co/t/recomendations-on-managing-storage-size/232905/1 "2020-05-16T06:27:36Z")

</div>

Hi,

I am new to Elasticsearch.

I have noticed that Elasticsearch is very generous when it comes to consuming storage. This makes it a challenge when the storage requirements start growing and the type of storage becomes costly (eg: SSD)  
Is there a way to manage this like with the use of compression?  
Does Elastic support compression methods?  
If so do you need licenses to do it or is it free?  
What is the best practice of keeping the storage at a manageable level ?

BR  
Rukmal

---

<div class="post-metadata">

### Author: ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [May 16, 2020, 6:34am UTC](https://discuss.elastic.co/t/recomendations-on-managing-storage-size/232905/2 "2020-05-16T06:34:43Z")

</div>

Have a look at [these docs](https://www.elastic.co/guide/en/elasticsearch/reference/7.7/tune-for-disk-usage.html) for guidance on reducing the size indices take up on disk.

---

<div class="post-metadata">

### Author: ![rukmalf](https://avatars.discourse-cdn.com/v4/letter/r/43a26b/32.png) [@rukmalf](https://discuss.elastic.co/u/rukmalf)
#### Post date: [May 17, 2020, 7:37am UTC](https://discuss.elastic.co/t/recomendations-on-managing-storage-size/232905/3 "2020-05-17T07:37:41Z")

</div>

Hi Christian,

Thank you for the reply. This is more towards getting rid of unwanted fields and stuff right? How about compression? Does Elastic have some sort of compression that we can use to reduce the size old data takes? I am coming from a SIEM background so hope you get what I am actually trying to do.

BR  
Rukmal

---

<div class="post-metadata">

### Author: ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [May 17, 2020, 8:35am UTC](https://discuss.elastic.co/t/recomendations-on-managing-storage-size/232905/4 "2020-05-17T08:35:13Z")

</div>

Elasticsearch compresses by default and the link I provided discusses the `best_compression` codec that further compresses the source. Making sure data is mapped efficiently and not using the default mappings unless necessary can as the docs describe substantially reduce the size on disk.

[This blog post](https://www.elastic.co/blog/filebeat-modiles-access-logs-and-elasticsearch-storage-requirements) discusses storage size and optimization. It is a bit old and refers to an older version of Elasticsearch (\_all field is e.g. no longer available in newer versions), but the principles are still valid.

---

<div class="post-metadata">

### Author: ![rukmalf](https://avatars.discourse-cdn.com/v4/letter/r/43a26b/32.png) [@rukmalf](https://discuss.elastic.co/u/rukmalf)
#### Post date: [May 17, 2020, 8:53am UTC](https://discuss.elastic.co/t/recomendations-on-managing-storage-size/232905/5 "2020-05-17T08:53:12Z")

</div>

Hi Christian,

Sorry if this question is stupid, But is there any indication on how much reduction can be achieved using the best\_compression codec?

---

<div class="post-metadata">

### Author: ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [May 17, 2020, 9:01am UTC](https://discuss.elastic.co/t/recomendations-on-managing-storage-size/232905/6 "2020-05-17T09:01:15Z")

</div>

It will depend a lot on the data and size of the documents and how large portion of the index size the source accounts for. For some sample data sets I have seen a reduction in storage size by around 20%, which roughly matches the data in the blog post I linked to.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [June 14, 2020, 9:01am UTC](https://discuss.elastic.co/t/recomendations-on-managing-storage-size/232905/7 "2020-06-14T09:01:17Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
