Hi, I am using ES 7.6.2 and I can not find the merge policy for my indices. I don't see it in _settings.
In addition, I can not find the default merge policy in the elastic documentation.
Can someone help me please?
The documentation on the merge policy is here.
The merge policy is the Tiered one.
It is not usually required to tweak settings which are not documented as they're usually necessary for special edge use cases (e.g. heavy delete use cases).
Other advanced settings are at:
- https://github.com/elastic/elasticsearch/blob/7b34e22890e15b7480689b216c48487c87385978/server/src/main/java/org/elasticsearch/index/MergeSchedulerConfig.java#L56
- https://github.com/elastic/elasticsearch/blob/7b34e22890e15b7480689b216c48487c87385978/server/src/main/java/org/elasticsearch/index/MergePolicyConfig.java#L49
I see that link but it doesn't has so much...
I have a time-based index containing APM transaction metrics (20GB, 2 shards) that isn't receiving data anymore, it is from yesterday.
When I inspect the segments, I can see that index has almost 100 segments.
Should I expect that Elasticsearch by itself merge that segments sometime?
Or should I execute a forcemerge to get only 1 or 2 segments for example?
No, do not force merge an index which you are indexing.
Only read only indices can be force merged without side effects. From the docs:
Running force merge against a read-write index can cause very large segments to be produced (>5Gb per segment), and the merge policy will never consider it for merging again until it mostly consists of deleted docs. >This can cause very large segments to remain in the shards.
Can you check the logs on Elasticsearch side or Apm server side to check if there are errors?
Are you using ILM? Maybe there's a problem in the index rollover.
Sorry, maybe I was not clear.
I'm not sending data to that index anymore, it is from yesterday.
Today metrics are being indexed in an index from today.
But I see a lot of segments in my old index (yesterday), and I don't know if Elasticsearch is going to merge that segments again sometime in the future.
Oh my bad! I misread your previous message!
Yes you can forcemerge.
If you're using ILM you could even embed the forcemerge in a policy to do it automatically.
1 Like
100 segments does not sound that much. Is this in any way causing problems?
Not really, but I would like to optimise the indices as much as possible, and I find that 1 segment per shard is the best for warm or cold indices...
I could execute a forcemerge, that's ok. But if not, should I expect that Elasticsearch by itself merge that segments sometime in the future? Or ES no longer works with that old indices anymore?
If you want to go to a single segment per shard you will need to manually forcemerge.
1 Like
Ok.. thank guys!
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.