In case of planned shutdown (curl -XPOST 'http://localhost:9200/_shutdown')
, is it possible to persist the RoutingTable in master node. (In case , we
are using separate master and data nodes.) So that when we start the
cluster node by node (starting from master) along with
gateway.expectednodes setting , then master node can re-use the persisted
RoutingTable instead of constructing new one (New one causing unnecessary
shard relocation, that too we have shards in big sizes). Once the expected
nodes are available in cluster reusing the same avoids unnecessary movement
of shrads with-in cluster.
there is a specific setting preventing shard allocation, see Elasticsearch Platform — Find real-time answers at scale | Elastic(the
disable_allocation setting). Alternatively you can also the gateway
recovery to start after a couple of minutes in addition to the expected
nodes, which might give you more time to fire up all nodes?
In case of planned shutdown (curl -XPOST 'http://localhost:9200/_shutdown')
, is it possible to persist the RoutingTable in master node. (In case , we
are using separate master and data nodes.) So that when we start the
cluster node by node (starting from master) along with
gateway.expectednodes setting , then master node can re-use the persisted
RoutingTable instead of constructing new one (New one causing unnecessary
shard relocation, that too we have shards in big sizes). Once the expected
nodes are available in cluster reusing the same avoids unnecessary movement
of shrads with-in cluster.
Thanks for replaying. gateway.expected_datanodes settings works nicely , it
starts recovery once all the datanodes (i configured) arrived.
Before performing full cluster restart , if the cluster in green state
,then no need for BalancedShardAllocator on initial recovery after restart
.There should be different recovery instead of normal procedure since
shards are already allocated and balanced before cluster restart.
If i understand correctly, BalancedShardsAllocator can be turned off
in-case of initial_recovery_from gateway, because while building
cachedShardsState in LocalGatewayAllocator->allocateUnassigned() , itself
we know the list of nodes which has the corresponding shard. Since initial
recovery index/search wont be possible at this stage, rather than going through
list of allocation deciders and BalancedShardsAllocator , we can directly
assign all the shards immediately to the corresponding nodes with out
throttling allocation. It avoids unnecessary shards movement across cluster due
to BalancedShardAllocator.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.