Old indexes still with many segments

Elastic Stack Elasticsearch
1

Hi ,
Our new index creation method is index per month ( so basically for 6 months we have 6 indexes ) . We keep 6 indexes available on the systems before we purge the old one , and only the newest is being active ingested (all other are use for search + the newest)

Looking into the index stats I still see hugh amount of segments for the old indexes . We trying to optimize our memory utilization by minimizing the amount of memory this old indexes consume (amount of segments ) , but seems like old indexes are not being auto optimizes / continuing being marge

Any configuration method exist to change this behave (We like to avoid force optimized , as we see that this process is consuming many resource ) ,

Example from our production system :

"myindex_2017_03"
segments" : {
"count" : 13003,
"memory_in_bytes" : 32476621590,
"index_writer_memory_in_bytes" : 0,
"index_writer_max_memory_in_bytes" : 194560000,
"version_map_memory_in_bytes" : 0,
"fixed_bit_set_memory_in_bytes" : 0
},

Thanks

2

That is exactly what force merge is for. You can control how many target segments you are merging into, so do not necessarily go all the way down to 1.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.