Compress Elasticsearch Index/Disk Usage Elasticsearch

Elastic Stack Elasticsearch
1

Hey guys,

I just wanted to ask if there´s a possibility to decrease the disk space
useage of an elasticsearch index.

e.g. i got an index called logstash-2014.03.17 with an size of 547MB and an
total of 1018761 indexed documents/logs and i want to reduce the disk usage
of it.

Regarding to that article i
found: https://github.com/elasticsearch/elasticsearch/issues/2037 all i
have to do is to set index.store.compress.stored to true.

I´ve done this with
-curl -XPOST 'localhost:9200/logstash-2014.03.17/_close'
-curl curl -XPUT 'localhost:9200/logstash-2014.03.17/_settings' -d '
{
"index" : {
" index.store.compress.stored" : true
} }
'
-curl -XPOST 'localhost:9200/logstash-2014.03.17/_open'

than i optimized the index with the command:

curl -XPOST 'http://localhost:9200/logstash-2014.03.17/_optimize'

and restarted the cluster.

Unfortunately nothing happend and it still uses 547MB of disk space.

Have i done sth. wrong?

Are there other ways to decrease the disk space useage?

Thanks for response.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/8b4d651e-7379-47e8-88a1-189308d118f3%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

2

Hi,

This setting doesn't exist anymore as indices are now always compressed.

One thing that might help gain back some (not much, don't expect too much
from it) disk is to run an _optimize[1] call on this index. You can use
curator[2] in order to automate this operation.

[1]

[2] GitHub - elastic/curator: Curator: Tending your Elasticsearch indices

On Wed, Mar 19, 2014 at 1:10 PM, horst knete baduncle23@hotmail.de wrote:

Hey guys,

I just wanted to ask if there´s a possibility to decrease the disk space
useage of an elasticsearch index.

e.g. i got an index called logstash-2014.03.17 with an size of 547MB and
an total of 1018761 indexed documents/logs and i want to reduce the disk
usage of it.

Regarding to that article i found:
Stored Fields Compression · Issue #2037 · elastic/elasticsearch · GitHub all i have to
do is to set index.store.compress.stored to true.

I´ve done this with
-curl -XPOST 'localhost:9200/logstash-2014.03.17/_close'
-curl curl -XPUT 'localhost:9200/logstash-2014.03.17/_settings' -d '
{
"index" : {
" index.store.compress.stored" : true
} }
'
-curl -XPOST 'localhost:9200/logstash-2014.03.17/_open'

than i optimized the index with the command:

curl -XPOST 'http://localhost:9200/logstash-2014.03.17/_optimize'

and restarted the cluster.

Unfortunately nothing happend and it still uses 547MB of disk space.

Have i done sth. wrong?

Are there other ways to decrease the disk space useage?

Thanks for response.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/8b4d651e-7379-47e8-88a1-189308d118f3%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/8b4d651e-7379-47e8-88a1-189308d118f3%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
Adrien Grand

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAL6Z4j7AeyNWuJJLDcadxVasEZQ%2Bu-M6eNJe2F7V7DjupZUSoA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

3

Hi Adrien,

thanks for respone.

The Curator is a very helpful tool thanks for the link!

I tried it and save about 1 GB of disk space (24->23GB) i think the effekt
increases with more indexes.

Am Mittwoch, 19. März 2014 17:42:56 UTC+1 schrieb Adrien Grand:

Hi,

This setting doesn't exist anymore as indices are now always compressed.

One thing that might help gain back some (not much, don't expect too much
from it) disk is to run an _optimize[1] call on this index. You can use
curator[2] in order to automate this operation.

[1]
Elasticsearch Platform — Find real-time answers at scale | Elastic
[2] GitHub - elastic/curator: Curator: Tending your Elasticsearch indices

On Wed, Mar 19, 2014 at 1:10 PM, horst knete <badun...@hotmail.de<javascript:>

wrote:

Hey guys,

I just wanted to ask if there´s a possibility to decrease the disk space
useage of an elasticsearch index.

e.g. i got an index called logstash-2014.03.17 with an size of 547MB and
an total of 1018761 indexed documents/logs and i want to reduce the disk
usage of it.

Regarding to that article i found:
Stored Fields Compression · Issue #2037 · elastic/elasticsearch · GitHub all i have to
do is to set index.store.compress.stored to true.

I´ve done this with
-curl -XPOST 'localhost:9200/logstash-2014.03.17/_close'
-curl curl -XPUT 'localhost:9200/logstash-2014.03.17/_settings' -d '
{
"index" : {
" index.store.compress.stored" : true
} }
'
-curl -XPOST 'localhost:9200/logstash-2014.03.17/_open'

than i optimized the index with the command:

curl -XPOST 'http://localhost:9200/logstash-2014.03.17/_optimize'

and restarted the cluster.

Unfortunately nothing happend and it still uses 547MB of disk space.

Have i done sth. wrong?

Are there other ways to decrease the disk space useage?

Thanks for response.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/8b4d651e-7379-47e8-88a1-189308d118f3%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/8b4d651e-7379-47e8-88a1-189308d118f3%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
Adrien Grand

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/814e94ff-50c8-42ba-b587-0f04b3a8812b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.