# Snapshot compress not compressing? **URL:** \ **Category:** Elasticsearch\ **Created:** [September 5, 2014, 9:27pm UTC](https://discuss.elastic.co/t/snapshot-compress-not-compressing/19650 "2014-09-05T21:27:09Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![ppearcy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ppearcy/32/980_2.png) [@ppearcy](https://discuss.elastic.co/u/ppearcy)\ **Post date:** [September 5, 2014, 9:27pm UTC](https://discuss.elastic.co/t/snapshot-compress-not-compressing/19650/1 "2014-09-05T21:27:09Z")
I am playing around with snapshot/restore and have a local 1.3.2 cluster running on Mac OS X with 894MB of index data. I have registered a backup repository like so (straight from the docs): curl -XPUT '[http://localhost:9200/\_snapshot/my\_backup](http://localhost:9200/_snapshot/my_backup)' -d '{ "type": "fs", "settings": { "location": "/tmp/backups/my\_backup", "compress": true } }' Then run the snapshot (again straight from the docs): curl -XPUT "localhost:9200/\_snapshot/my\_backup/snapshot\_1?wait\_for\_completion=true" The snapshot runs fine, but the backup directory that is generated is 890MB, which tells me that compression isn't kicking in. When I set compress: false, I get the same results. If I tar/gz that directory it gets squashed down to 204MB. I'd expect the compressed snapshot from ES to be somewhere in that ballpark. Am I doing something wrong or is there a bug? Thanks and Best Regards, Paul -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/93e37247-58b4-4d08-bef9-8de0146cf979%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/93e37247-58b4-4d08-bef9-8de0146cf979%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Russell\_Seymour](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/russell_seymour/32/1298_2.png) [@Russell\_Seymour](https://discuss.elastic.co/u/Russell_Seymour)\ **Post date:** [September 8, 2014, 9:22am UTC](https://discuss.elastic.co/t/snapshot-compress-not-compressing/19650/2 "2014-09-08T09:22:09Z")
Good morning, I experienced the exact same issue on Friday as well. I have an Elastic Search cluster (1.3.2) running on Windows using Oracle Java 1.7.0\_67. We needed a backup strategy and purposely upgraded to this version to take advantage of the snapshots feature. The size of the indexes in the cluster is about 40Gb and even with the 'compress' option explicitly set to true (as in Paul's post and in the documentation) the snapshot is about 40Gb, Is there a work around to get this working or some other fix? Thanks, Russell On Friday, 5 September 2014 22:27:09 UTC+1, ppearcy wrote: > I am playing around with snapshot/restore and have a local 1.3.2 cluster > running on Mac OS X with 894MB of index data. > > I have registered a backup repository like so (straight from the docs): > > curl -XPUT '[http://localhost:9200/\_snapshot/my\_backup](http://localhost:9200/_snapshot/my_backup)' -d '{ > "type": "fs", > "settings": { > "location": "/tmp/backups/my\_backup", > "compress": true > } > }' > > Then run the snapshot (again straight from the docs): > > curl -XPUT > "localhost:9200/\_snapshot/my\_backup/snapshot\_1?wait\_for\_completion=true" > > The snapshot runs fine, but the backup directory that is generated is > 890MB, which tells me that compression isn't kicking in. When I set > compress: false, I get the same results. > > If I tar/gz that directory it gets squashed down to 204MB. I'd expect the > compressed snapshot from ES to be somewhere in that ballpark. > > Am I doing something wrong or is there a bug? > Thanks and Best Regards, > Paul -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/0c02d519-efc8-4dff-91a8-068998052187%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/0c02d519-efc8-4dff-91a8-068998052187%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Igor\_Motov](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/igor_motov/32/45193_2.png) [@Igor\_Motov](https://discuss.elastic.co/u/Igor_Motov)\ **Post date:** [September 8, 2014, 6:00pm UTC](https://discuss.elastic.co/t/snapshot-compress-not-compressing/19650/3 "2014-09-08T18:00:29Z")
At the moment, compression is applied only to metadata files (index mapping and settings basically). Data files are not compressed. On Monday, September 8, 2014 5:22:09 AM UTC-4, Russell Seymour wrote: > Good morning, > > I experienced the exact same issue on Friday as well. > > I have an Elastic Search cluster (1.3.2) running on Windows using Oracle > Java 1.7.0\_67. We needed a backup strategy and purposely upgraded to this > version to take advantage of the snapshots feature. > The size of the indexes in the cluster is about 40Gb and even with the > 'compress' option explicitly set to true (as in Paul's post and in the > documentation) the snapshot is about 40Gb, > > Is there a work around to get this working or some other fix? > > Thanks, Russell > > On Friday, 5 September 2014 22:27:09 UTC+1, ppearcy wrote: > > > I am playing around with snapshot/restore and have a local 1.3.2 cluster > > running on Mac OS X with 894MB of index data. > > > > I have registered a backup repository like so (straight from the docs): > > > > curl -XPUT '[http://localhost:9200/\_snapshot/my\_backup](http://localhost:9200/_snapshot/my_backup)' -d '{ > > "type": "fs", > > "settings": { > > "location": "/tmp/backups/my\_backup", > > "compress": true > > } > > }' > > > > Then run the snapshot (again straight from the docs): > > > > curl -XPUT > > "localhost:9200/\_snapshot/my\_backup/snapshot\_1?wait\_for\_completion=true" > > > > The snapshot runs fine, but the backup directory that is generated is > > 890MB, which tells me that compression isn't kicking in. When I set > > compress: false, I get the same results. > > > > If I tar/gz that directory it gets squashed down to 204MB. I'd expect the > > compressed snapshot from ES to be somewhere in that ballpark. > > > > Am I doing something wrong or is there a bug? > > Thanks and Best Regards, > > Paul -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/ab00335a-8216-45df-ab8e-09b33086ecaf%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/ab00335a-8216-45df-ab8e-09b33086ecaf%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![ppearcy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ppearcy/32/980_2.png) [@ppearcy](https://discuss.elastic.co/u/ppearcy)\ **Post date:** [September 9, 2014, 4:20am UTC](https://discuss.elastic.co/t/snapshot-compress-not-compressing/19650/4 "2014-09-09T04:20:14Z")
Hehe, good to know. I submitted a PR to clarify the documentation: > > > Clarify s3 snapshot compress behavior only applies to metadata and no index file…s. The "at the moment" leads me to believe this is planned or in the pipeline, looking forward to it. Best Regards, Paul On Monday, September 8, 2014 2:00:30 PM UTC-4, Igor Motov wrote: > At the moment, compression is applied only to metadata files (index > mapping and settings basically). Data files are not compressed. > > On Monday, September 8, 2014 5:22:09 AM UTC-4, Russell Seymour wrote: > > > Good morning, > > > > I experienced the exact same issue on Friday as well. > > > > I have an Elastic Search cluster (1.3.2) running on Windows using Oracle > > Java 1.7.0\_67. We needed a backup strategy and purposely upgraded to this > > version to take advantage of the snapshots feature. > > The size of the indexes in the cluster is about 40Gb and even with the > > 'compress' option explicitly set to true (as in Paul's post and in the > > documentation) the snapshot is about 40Gb, > > > > Is there a work around to get this working or some other fix? > > > > Thanks, Russell > > > > On Friday, 5 September 2014 22:27:09 UTC+1, ppearcy wrote: > > > > > I am playing around with snapshot/restore and have a local 1.3.2 cluster > > > running on Mac OS X with 894MB of index data. > > > > > > I have registered a backup repository like so (straight from the docs): > > > > > > curl -XPUT '[http://localhost:9200/\_snapshot/my\_backup](http://localhost:9200/_snapshot/my_backup)' -d '{ > > > "type": "fs", > > > "settings": { > > > "location": "/tmp/backups/my\_backup", > > > "compress": true > > > } > > > }' > > > > > > Then run the snapshot (again straight from the docs): > > > > > > curl -XPUT > > > "localhost:9200/\_snapshot/my\_backup/snapshot\_1?wait\_for\_completion=true" > > > > > > The snapshot runs fine, but the backup directory that is generated is > > > 890MB, which tells me that compression isn't kicking in. When I set > > > compress: false, I get the same results. > > > > > > If I tar/gz that directory it gets squashed down to 204MB. I'd expect > > > the compressed snapshot from ES to be somewhere in that ballpark. > > > > > > Am I doing something wrong or is there a bug? > > > Thanks and Best Regards, > > > Paul -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/e71eb039-3f13-4417-8427-eace733bca53%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/e71eb039-3f13-4417-8427-eace733bca53%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 1:03am UTC](https://discuss.elastic.co/t/snapshot-compress-not-compressing/19650/5 "2017-07-06T01:03:48Z")