i tried changing the index.codec on a existing index to "best_compression"(DEFLATE) to achieve better compression values, than the standard LZ4.
I closed the index, changed the codec, reopened it and than run the optimize api on it with setting the max_num_segments to 20.
The index did have 998 GB (according to kopf-plugin), after the optimize command finished (took ages, somehow the merging from _optmize only run with 20MB/s) the index still got 996 GB according to kopf.
Is it in the nature of ES that existing indices can not be compressed very good? Can i expect better resuls when setting index.codec at index creating time?
I am tryting to replace the compression of our filesystem with this, since the CPU overhead is quite large now and i think that the performance decrease from ES will not have that big of an impact.
Thanks for your response.