I do not know. Have not used Elasticsearch 2.x in a long time...
In your case seems to be related to https://github.com/elastic/elasticsearch/pull/29125 which was fixed in version 6.2.4.
We also noticed a huge increase in open file descriptors:
Hi, @Wedney_Yuri I think the issue you described is related to https://github.com/elastic/elasticsearch/issues/29488.
We are experiencing a similar issue with 6.2.4 as described in https://github.com/elastic/elasticsearch/issues/30351
Hopefully, the following merge that will be release in 6.3.0 will solve the issue:
Thank you, I will investigate.
You're right, we're noticing full disk usage in version 6.2.4. How did you solve this problem?
As far as I know there is no solution and we are left to wait for 6.3.0 to be released.
Calling force merge didn't help. The only two things that cause the shards to shrink were:
- Closing and reopen the index
- Restart the node that the shards is on
These actions will force elastic to initialize the shards and only then it will run a successful merge on these shards.
Both are not a valid workaround 
Note that the initialize of these shards can take ages depends on the shard size, during this time the data in these shards won't be accessible.
1 Like
It should be sufficient to set the number of replicas to zero and back to 1, which will cause the replicas to be reinitialised. It is also even possible to use the reroute API to cancel the allocation of the problematic replica which will cause just that one shard to be rebuilt.
In our case, the shards that didn't perform merge operations were the primaries and removing the replicas didn't do the trick.
1 Like
As @farin99 said, it's happening to primary nodes as well.
This has been happening almost every day, do you know which is the last stable version of elasticsearch that was released without this bug?
@farin99 I don't know why, but we changed indices.recovery.max_bytes_per_sec to 4000mb and the disk usage was reduced after minutes.
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.