Remove Node From Cluster?

[Mike_Thoma]

Good day,

While putting together a POC for a client, a team member accidentally terminated a node in AWS. My cluster health is "red".

I see how you can exclude a node by IP, but the IP address was lost.

What is the most graceful way of removing the node if all I know is the node name?

Thanks for your time.
Mike

[xavierfacq]

Yes see Routing : https://www.elastic.co/guide/en/elasticsearch/reference/5.0/allocation-filtering.html

# Routing at Cluster level on attribute
curl -XPUT 'http://yourserver:9200/_cluster/settings?pretty' -d '
{
  "transient" : {
    "cluster.routing.allocation.exclude._name" : "node_name_to_exclude"
  }
}'

[xavierfacq]

Note that I recently have this problem: If you have mixed node versions and if shards have moved to a newer version, it cannot be re-moved to an older version node... (Mixing Version 2.2.0 and 2.4.1)

[Mike_Thoma]

Xavier you are the man!! Thank you! I will keep the version info in mind.

I deleted the node, added a new one, but the status is still red due to 3 unassigned shards. I have it set to no replicas, so not sure what's happening. Figured I'd check back for some help before I break things worse. I am still learning the basics if you couldn't tell.

{
"cluster_name" : "POC2",
"status" : "red",
"timed_out" : false,
"number_of_nodes" : 2,
"number_of_data_nodes" : 2,
"active_primary_shards" : 3,
"active_shards" : 3,
"relocating_shards" : 0,
"initializing_shards" : 0,
"unassigned_shards" : 3,
"delayed_unassigned_shards" : 0,
"number_of_pending_tasks" : 0,
"number_of_in_flight_fetch" : 0,
"task_max_waiting_in_queue_millis" : 0,
"active_shards_percent_as_number" : 50.0
}

[Mike_Thoma]

My issue with reassigning was due to not using the "allow_primary" flag on the API. Fixed the unassigned using this:

curl -XPOST "http://localhost:9200/_cluster/reroute" -d'
{
"commands" : [ {
"allocate" : {
"index" : ".kibana", "shard" : 0, "node" : "node-1", "allow_primary": "true"
}
}
]
}'

[xavierfacq]

Good ! I hope everthing is Green now

[Mike_Thoma]