Rollover and Consolidation of Shards

Elastic Stack Elasticsearch
1

Hello, my cluster has 1150 primary shards with 2300 total shards. I wanted to:

  1. consolidate them into recommended size of 50 GB each (100 with replica). I have attempted to this via Kibana GUI (using force merge) but it doesn't work. There is no error and I do get a "successfully merged" feedback. However, I can the individual shards.
  2. I also want to ILM policy for them to only rollover once they reach 50 GB. I've tried to set this manually but it keeps failing. Could someone please assist?

I have two data nodes and one voting only node.

{
    "cluster_name": "data_analytics_1",
    "status": "green",
    "timed_out": false,
    "number_of_nodes": 3,
    "number_of_data_nodes": 2,
    "active_primary_shards": 1150,
    "active_shards": 2300,
    "relocating_shards": 0,
    "initializing_shards": 0,
    "unassigned_shards": 0,
    "delayed_unassigned_shards": 0,
    "number_of_pending_tasks": 0,
    "number_of_in_flight_fetch": 0,
    "task_max_waiting_in_queue_millis": 0,
    "active_shards_percent_as_number": 100.0
}
2

Forcemerge does only merge segments within a shard and does not reduce shard count. If you have indices with more than 1 primary shard you can look at the shrink index API.

If your indices have a single primary shard you will need to reindex your data into new indices in order to combine indices/shards.

Please show what you have tried and elaborate on what did not work.

1 Like
3

Thanks a lot @Christian_Dahlqvist , let me try the first two options and revert.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.