Index size is increasing during the forcemerge

Elastic Stack Elasticsearch
1

Hi Elastic team,

I'm using hot-warm-cold ILM to manage my indexes.

However, in forcemerge action, I'm curious why the index size is increasing during the forcemerge.

From the picture below, the primary shard size is only 238.2GB but the store size is 746.8GB!!!.

index
index.png1304×562 48.4 KB

Furthermore, it tooks a very long time in forcemerge step (it still not finish).

Example task here.

      "wp1XHBmkTmGgotuC0Autiw:16080117" : {
          "node" : "wp1XHBmkTmGgotuC0Autiw",
          "id" : 16080117,
          "type" : "transport",
          "action" : "indices:admin/forcemerge",
          "description" : "",
          "start_time_in_millis" : 1568122238700,
          "running_time_in_nanos" : 79752320903239,
          "cancellable" : false,
          "headers" : { }
       },

Could you please help to explain this behaviour?

Thank you

2

A merge writes out a fresh copy of all the data it's merging and then cleans up the old segments that are no longer needed. If you are force-merging to a single segment then that means it must write out a fresh copy of the entire shard. Only once it's finished making a new (huge) segment can it clean up the old segments.

3

Thank you. How do I track the forcemerge progress? How can I make it faster? Does the increasing of " indices . memory . index_buffer_size" help the speed?

4

There isn't much you can do to tune the speed of a merge. It's quite an expensive and complicated operation and on a large shard it can take a long time to complete.

1 Like
5

Thank you. It was old one. I increased the number of shards for the new one.

Will more shards help the indexing performance? From my understanding, it helps only the search.

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.