Using /_cluster/settings -> cluster.routing.allocation.exclude works for only one node?

[preddy75 (Scott Preddy)]

Hi,

I am running ELK on large Kubernetes environment, and it is common case to bring nodes in and out of environment. Right now we have mixed sized disks and need to route away from small nodes to large nodes.

I am issuing:

PUT /_cluster/settings
{
"transient": {
"cluster.routing.allocation.exclude._name": "node-1"
}
}

I come back later and "node-1" is drained down to 1 shard (searchguard). I want to take out another node as well before scaling down my Elasticsearch statefulset to permanently pull these two nodes out.

However after I issued:

PUT /_cluster/settings
{
"transient": {
"cluster.routing.allocation.exclude._name": "node-2"
}
}

I noticed "node-1" was no longer in my settings (which now that I am looking at rest of API makes sense)

{
"persistent": {
"cluster": {
"routing": {
"allocation": {
"cluster_concurrent_rebalance": "16",
"node_concurrent_recoveries": "4",
"node_initial_primaries_recoveries": "8",
"enable": "all"
}
}
}
},
"transient": {
"cluster": {
"routing": {
"allocation": {
"exclude": {
"_name": "node-2"
}
}
}
}
}
}

So of course when I came back the next day, node-1 was full again, and node-2 was drained down to searchguard.

I see in the documentation that you all a "*" in the node name, but that won't allow me to accomplish what I want, keeping "node-1", and "node-2" in drained state and not being routed to before I shutdown ES.

Namely,
https://www.elastic.co/guide/en/elasticsearch/reference/5.6/allocation-filtering.html

Is there a way to specify more than one node without using just *?

[DavidTurner]

Yes, you can use a comma-separated list of node names:

PUT /_cluster/settings
{
  "persistent": {
    "cluster.routing.allocation.exclude._name":"node-1,node-2"
  }
}

[preddy75 (Scott Preddy)]

Thank you. Works.

[preddy75 (Scott Preddy)]

And btw, for those running in env's that have nodes restart from time to time, make sure its persistent (like solution says).