if I have index size of 2tb and node size is 1tb , In that case how index will split on another nodes and if i add new data nodes so how this data can be distribute, it will do automatically? Or I can do it manually. I want horizontal scaling not vertical scaling . So this index will split into another nodes rather than node failure?
How many primary and replica shards does this index have? Elasticsearch will automatically redistribute shards if you add nodes, but will not automatically split shards if you do not have enough.
if I set 1 primary and 1 replica shard, what happened in this case
If you only have a single index with 2 shards in total the data can not be distributes across more than 2 nodes (one shard per node). In order to scale out you would need to use the split index API to increase the number of primary shards. Given the size of your index this is probably wise anyway as your shards are currently huge.
So If data nodes are full , how to avoid unassigned shards and cluster health also must be green?
If you have a reasonable shard size, e.g. less than 50GB, you just add nodes and there are enough shards so that Elasticsearch can rebalance.
If you only have a couple of exceptionally large shards and you are running out of disk space you will need to split the index. This does unfortunately require a certain amount of free disk space, so may not be possible unless you are able to increase the amount of disk on the nodes.
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.