# Elasticsearch Compression ratio **URL:** \ **Category:** Elasticsearch\ **Created:** [July 17, 2017, 3:48pm UTC](https://discuss.elastic.co/t/elasticsearch-compression-ratio/93424 "2017-07-17T15:48:47Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![hossein\_ey](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hossein_ey/32/20117_2.png) [@hossein\_ey](https://discuss.elastic.co/u/hossein_ey)\ **Post date:** [July 17, 2017, 3:48pm UTC](https://discuss.elastic.co/t/elasticsearch-compression-ratio/93424/1 "2017-07-17T15:48:47Z")
we want to use Elasticsearch for a large amount of data. one of the important issue is storage usage. we create a sample index with 412 million ![](https://us1.discourse-cdn.com/elastic/original/3X/a/c/acb4ab07ba8af9b48e56c91d1f350986d6258524.PNG) 412 million rows take 242 GB of hard ===\> 590 Bytes for each row we know each row of our data with json format has 800-1000 Bytes size so elastic compressed our 900 Bytes data into 580 Bytes ... is there any better way to compress our data? ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [July 17, 2017, 3:53pm UTC](https://discuss.elastic.co/t/elasticsearch-compression-ratio/93424/2 "2017-07-17T15:53:59Z")
There are some guidelines [here](https://www.elastic.co/guide/en/elasticsearch/reference/5.5/tune-for-disk-usage.html#_use_literal_best_compression_literal). What does your mapping look like? ---
**Author:** ![hossein\_ey](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hossein_ey/32/20117_2.png) [@hossein\_ey](https://discuss.elastic.co/u/hossein_ey)\ **Post date:** [July 17, 2017, 4:22pm UTC](https://discuss.elastic.co/t/elasticsearch-compression-ratio/93424/3 "2017-07-17T16:22:32Z")
most fields are integer... we have 50-60 fields for each document and most of them should be searchable( exact search and range search ) ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [July 17, 2017, 4:31pm UTC](https://discuss.elastic.co/t/elasticsearch-compression-ratio/93424/4 "2017-07-17T16:31:03Z")
Are you using the `best_compression` codec? Do you have the `_all` field enabled? If so, do you need it? ---
**Author:** ![hossein\_ey](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hossein_ey/32/20117_2.png) [@hossein\_ey](https://discuss.elastic.co/u/hossein_ey)\ **Post date:** [July 18, 2017, 6:49am UTC](https://discuss.elastic.co/t/elasticsearch-compression-ratio/93424/5 "2017-07-18T06:49:28Z")
we use default settings of Logastash template And Elastic ... the compression is default( I think lz4) and \_all filed is enabled. we don't need \_all field and we should disable it ... but for best\_compression, does it impact the indexing performance? ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [July 18, 2017, 6:59am UTC](https://discuss.elastic.co/t/elasticsearch-compression-ratio/93424/6 "2017-07-18T06:59:24Z")
Using `best_compression` does have some impact on indexing performance, but does compress the source a lot better and can save a significant amount of disk space. Disabling the `_all` field will also save space if you do not need it. If you have fields that do not need to be aggregated upon or be subject to free-text search, you can also slimline the default Logstash mappings and not have all fields dual-mapped. I discussed these topics and the trade-offs in [blog post](https://www.elastic.co/blog/filebeat-modiles-access-logs-and-elasticsearch-storage-requirements). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [August 15, 2017, 6:59am UTC](https://discuss.elastic.co/t/elasticsearch-compression-ratio/93424/7 "2017-08-15T06:59:51Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.