I am more of a developer than ops, so I am wondering how to deploy a HA cluster on AWS' EC2 instances. The thing is, I want this cluster to survive a zone failure...
Here's what I have planned so far:
- Across 2 availability zones (say, A and B)
- Each node will be properly configured with an attribute that indicates to which zone it belongs
- Cluster and indices will be aware of that, so it can allocate replicas to the other zone
- I was thinking of having 3-node in each zone, so if one zone completly fails, the election between the three remaining node is successeful
- Each node will act as master eligible and data
The questions I have now:
- Do we still have to have at least 3 node to avoid split brain? (I didn't kept up to date with some change in > 7 that I think solved that)
- In this scenario the total 6 node must be working all the time. Is there a more cost effective way to assure the HA cross availability zones?
- How should I set the network configuration for the discoery phase? Install and use the EC2 plugin?
- And what about the clients? How should they be configured to access this cluster in a HA fashion? Should I list all the nodes IPs in the "hosts" attribute, so in case of a failure it trys the next one?
- Or should I have some balancer in front of the cluster?
I'll appreciate any info you can give me. If there's already a post, an Elastic's blog or an article giving guidelines around this scenario, please send them my way.
As I said, I more used to writinf queries than to deploying clusters...