I have 2 zones, and want my elastic cluster to scale across them but for some reason ECE does not let me choose the 2 zones option with a Not available when you only have two zones. See Fault Tolerance in the documentation to learn more.
Also is there a way to change what zone an ece instance is allocated to after the initial creation ?
Fault tolerance
1 zone — Great for testing and development
2 zones — For production use
Not available when you only have two zones. See Fault Tolerance in the documentation to learn more.
Yes, this behavior is by design and meant to prevent a split brain scenaio. When selecting 2 availability zones ECE still requires to have 3 availability zones available to spin a third tie breaker node, that will not store any data, but will be used for master selection if needed.
We are working on natively supporting 2 availabillity zones installations and it should be available shortly. As an intermidiate solution, you can workaround this issue by creating a third "dummy" zone and tag the allocator in that zone with "elasticsearch_data": false to ensure that this allocator will only be used for master eligable tie breaker nodes, preventing it from hosting data nodes. This can be done using the following REST api:
There is an API to vacate an allocator, which will trigger migration of all Elasticsearch nodes, and Kibana instances deployed on that allocator to another one, while still aligning with any availability zones configuration. There is also an option to move nodes of a specific cluster.
There is, currently, no way to control the specific allocator to which an Elasticsearch node, or Kibana instance will be deployed, so if you have 3 AZs with 3 Allocators in each one, and you created a deployment with 2 AZs vacating a specific Allocator in AZ 1 doesn't mean that the nods running in that allocator will be migrated to another AZ. ECE algorythem will aim to fill Allocators while respecting the AZ configuration, but this means that the nodes might migrate to another allocator in the same AZ.
Having said that, we are working on a mechanism to allow ECE admin control over mapping between clusters and node types to specific allocators.
Thanks for the note above, that jives with my understanding of how allocation works, I was actually asking about changing assigned avaialabilty-zone on a runner/allocator from us-east-2a to us-east-2b after the initial install of ECE bits on it.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.