How to restart an elastic cluster?


(Anton Alfred) #1

Have tried to follow the following articles.
Talking about Rolling Updates

Talking about Full Restart

While these are predominantly based on upgrades, I have a requirement of restarting an elastic search.

What we need is restart the nodes in a cluster one by one without any of the shards moving or synchronizing with the replica.

We stopped processing so there are no changes happening on the nodes.

We have a primary and one replica shard for each index.

We tried these settings
"cluster.routing.allocation.enable": "none"

But this only talks about the primary shard not moving as soon as we enable it after restart the replica shard movement starts.

Is there a setting that just restarts elastic and attaches to the same primary and replica shards present locally. We already know there is no change to the data. All we need is elastic to restart and get a fresh start.

We do not want both relocating and initializing (not sure what does initializing mean)


(Mark Walkom) #2

What version are you on?


(Anton Alfred) #3

We are using 5.2 but we are ok to give the answer in latest version ?


(Mark Walkom) #4

If you follow the rolling restart docs then this should happen. Is that not what you are seeing?


(Anton Alfred) #5

Thanks for the reply. We figured out the issue. The issue was the settings were misplaced in the document.

We did the setting for one node, when we did for next node, since the setting was back on the persistent setting and we already had a transient setting overriding it, it was not taking effect.

{
  "persistent": {
    "cluster.routing.allocation.enable": "none"
  }
}

{
  "transient": {
    "cluster.routing.allocation.enable": "all"
  }
}

The document should be updated to do both settings either in persistent mode or in transient mode but not a mix of both when we have more than 1 node.

{
  "transient": {
    "cluster.routing.allocation.enable": "none"
  }
}

{
  "transient": {
    "cluster.routing.allocation.enable": "all"
  }
}

With the above setting we were able to do a simple restart of each node in a cluster.

So the process for a N Node cluster
For a Node 1

  1. Disable Allocation
  2. Restart the node (The node joins the cluster, all the shards on the node become in unassigned state)
  3. Enable Allocation
  4. Wait for unassigned shards both primary and replica to be allocated (Check no unassigned shards and cluster health green). The rest of the document is not correct, no more movement happens. The node would now have all the local shards assigned to it.

Depending on the shard count and size of each shard the initialization could take minutes in our case 5 minutes. Do not restart any other node before all shards are initialized.

Do the same for each of the other nodes.


(system) #6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.