We have been force merging our data, however we recently learned about setting "index.codec" to "best_compression" to potentially gain space from an index.
However since we have already force_merged to 1 segment per shard, we cant force_merge again to take advantage of the codec.
We are on 5.2.1. My friend who told me about "index.codec" is on 2.3.x and said they were able to rerun forcemerging even though the index had been merged down to 1 segment/shard.
I tried raising the max_num_segments but that didnt work, since it was the max, not the min.
Hey Mark,
I did rerun it with max_num_segments=1 and it returned instantly. I figured it would have tried to recompress though?
Order of events:
_forcemerge?max_num_segments=1 on index (this took some time and was run some time ago)
close index
set index.codec to best_compression
open index
rerun _forcemerge?max_num_segments=1 on index (returns instantly, no change to index size or anything else)
I was expecting something to happen on the second running of the forcemerge. My friend said their indices fully reran and shrunk by at least 20%. I know everyones clsuter and data is different. But expected something to happen lol.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.