Compress ELK data node


(Asaf Izak) #1

hello all

there is option in node level or index level to configure
elasticsearch data compress

thx
asaf


(David Pilato) #2

Look at https://www.elastic.co/guide/en/elasticsearch/reference/2.4/index-modules.html#_static_index_settings

index.codec

(Asaf Izak) #3

so as i undrostood i've just need to add index.codec: best_compression to elasticsearch.yml file


(Mark Walkom) #4

You can apply that to the index dynamically.


(Shahar Mor) #5

Can this be done in 2.4? Its listed under static settings in the docs and I'm getting this error when trying:
Can't update non dynamic settings[[index.codec]] for open indices


(Mark Walkom) #6

You do need to close the index, apply, then reopen it.


(Shahar Mor) #7

Got it.
If i want to both force merge & change the codec, should i first change the codec and then merge, or first merge and then change the codec?


(Mark Walkom) #8

EDIT - actually I was wrong. You need to change the codec first because the merge is what applies it.


(Shahar Mor) #9

And if i already merged to 1 segment, will running the merge again do something? Because when I'm running it now, it finishes immediately. So how can i change the codec now?


(Mark Walkom) #10

You can change it, just run a force merge again.


(Shahar Mor) #11

It does not re-run the merge once its fully merged to 1 segment per shard.


(Daniel Penning) #12

Don't expect huge savings from doing this. The index should already be compressed. You are just setting a higher compression level.


(Shahar Mor) #13

I reached 50% savings, thats very huge for me :slight_smile:


(Shahar Mor) #14

For tracking purposes: https://github.com/elastic/elasticsearch/issues/20936


(system) #15