Dynamically adding replicas to index

(Pavel Pavelsky) #1

Is it possible to dynamically add as many replicas to my cluster as I want?
My setup is: 1 index with 1 shard and 1 replica, and in case of high load I want to add more replicas to handle more requests. I don't know ahead how many replicas I will need.

For now I only see that I can define number of replicas when I create the index. I can also increase number of replicas using REST API, but I want to have it fully automatic, I can't have any steps before starting up a new server (I'm using Amazon's Autoscaling mechanism of EC2).

(Colin Goodheart-Smithe) #2

You can use the REST API as you mentioned to increase the number of replicas (or decrease it) dynamically, but you would need orchestration logic somewhere outside of Elasticsearch that determine that you need to scale up, provisions new VMs, installs Elasticsearch (may by using the puppet modules or chef cookbooks), starts Elasticsearch on your new VMs and then calls to the REST API to increase the number of replicas. You would also need similar logic to decommission nodes that you no longer require (basically do the reverse of above). This is all certainly possible but there is nothing currently in Elasticsearch to perform those steps for you.

(Mark Walkom) #3

You could probably build something with a Watcher webhook to do this.

(system) #4