Do I need to wait for shard allocation to complete or should I simply check if cluster status is green and then move over to the next node?
In addition, do I also need to worry about initializing shards before restarting another node?
Say, I only wait for cluster to become green and try to restart a node that has a primary and its replica shard is still in the process of relocating. Will I run into any issues? How does elasticsearch handle such a scenario?
When I have to do rolling restart, currently, I check if shard relocation is complete before I switch over to restart another node.
Why are shards being reallocated? The recommendation is to disable allocations while a node is being restarted and enable it once it's up again, so it should return to green very quickly without any additional reallocations.
Do I need to wait for shard allocation to complete or should I simply check if cluster status is green and then move over to the next node?
Since you have two replicas of each shard you don't even have to wait for the cluster to become green since you can handle two unavailable nodes at a time. But sure, taking them down one at a time and only when the cluster is green means you could suffer an unplanned node loss during the restart without venturing data availability.
Why are shards being reallocated? The recommendation is to disable allocations while a node is being restarted and enable it once it's up again, so it should return to green very quickly without any additional reallocations.
Right... before I shutdown, I do disable shard allocation by executing the command:
Once, allocation is enabled, I see that shards start getting re-allocated. I have seen that cluster goes green but the shards are still being relocated to different machines. And that's where my question is: Do I need to wait for the shard relocation to complete before I move onto the next node or not?
Once, allocation is enabled, I see that shards start getting re-allocated. I have seen that cluster goes green but the shards are still being relocated to different machines.
So the reallocation starts after the cluster goes green when you've restarted a single node? That's not what I would've expected—that it thinks there's a need to reallocate, that is. Unless things have happened while the node was down the previous shard equilibrium should remain.
And that's where my question is: Do I need to wait for the shard relocation to complete before I move onto the next node or not?
So the reallocation starts after the cluster goes green when you've restarted a single node? That's not what I would've expected—that it thinks there's a need to reallocate, that is. Unless things have happened while the node was down the previous shard equilibrium should remain.
I think I understand now. Yes, when the node was down, indexing was still going on. Thus, when the node restarts, the shards get re-allocated on enabling reallocation setting.
I have made appropriate changes that confirms that cluster is green before restarting the next node.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant logo are trademarks of the Apache Software Foundation in the United States and/or other countries.