I want the opinion about scaling down elasticsearch.
I have 2 shard cluster having 2 replicas (2P + 4R = 6shards) running on kubernetes. Now I want to remove replicas and corresponding nodes. I reduced kubernetes deployment pods but sometimes it deletes nodes having Primary replicas.
How to handle this scenario ? What is your opinion on following approach:
- Get nodes that have primary shards using
/_cat/shards
api. - Reduce nodes and during the graceful shutdown, if kubernetes deletes nodes with primary then move the primary shards using
_cluster/reroute
api. https://www.elastic.co/guide/en/elasticsearch/reference/6.8/cluster-reroute.html
During shutdown of a node, I already decommission node as a part of shut down.
Any help is appreciated.
Thanks