Shard rebalancing vs shard allocation

Elastic Stack Elasticsearch
1

Hi,

What is the difference/link between shard rebalancing and shard allocation ?

Is it possible to have a shard rebalancing with a disable sharding allocation ? I don't think so but I need some confirmation.

Furthermore, in the elastic rolling upgrade procedure (https://www.elastic.co/guide/en/elasticsearch/reference/current/rolling-upgrades.html), it is advised to disable shard allocation. My guess is that this step is necessary to prevent the cluster from creating new shards in order to satisfy the number of shards specified for the indices.

I have already read this topic (Difference between 'rebalance.enable' and 'allocation.enable') but I find the explanation quite vague.

Thanks for your help,

2

Shard allocation is about taking unassigned shard copies and finding each of them a node to inhabit. Rebalancing is about taking assigned shard copies and moving them to different nodes to even out the distribution of shards.

You are correct - for rebalancing to move a shard between nodes it needs to allocate the shard on the target node, which isn't allowed if allocation is disabled.

Possibly, but this isn't the terminology we normally use. You can create new shard copies by (a) creating an index or (b) increasing the number of replicas of an existing index, but I think this isn't what you meant by "creating new shards": I think you were talking about assigning unassigned shards.

Allocating a shard to a node involves copying its data from the primary, which could be an expensive and time-consuming operation. During a rolling restart this is unnecessary because each node is only temporarily unavailable, so it's better to leave the shards where they are until the node rejoins the cluster.

1 Like
3

Thank you very much for this very good answer !

Possibly, but this isn't the terminology we normally use. You can create new shard copies by (a) creating an index or (b) increasing the number of replicas of an existing index, but I think this isn't what you meant by "creating new shards": I think you were talking about assigning unassigned shards.

And you are absolutely right: I was talking about unassigned shards.

Have a good day,

1 Like
4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.