We just use plain old zen, but have some startup code that sets the aws
zone into an environment variable which we use only for balance allocation
awareness.
EC2 module seemed like overkill or maybe underkill because it didn't do
what we wanted. Unicast only has to reach one of the hosts in the cluster,
so we list a host from each zone. That way if any zone at all is up, all
the machines can find each other.
That is, if you know you're going to have at least 3 nodes in your cluster,
you can just list those 3 nodes in your config, and no matter how large
your cluster is, the cluster will find those nodes and bootstrap.
On Fri, Aug 23, 2013 at 12:55 PM, Pierce Wetter obastard@gmail.com wrote:
We just use plain old zen,
I still haven't given up on the ec2 plugin; I replace all machines
quite often (incl each time I do a small update), so having an
automatic discovery mechanism simplifies things a lot! But if I can't
find proper documentation and get it to work more reliably, I'll need
to reconsider...
but have some startup code that sets the aws zone
into an environment variable which we use only for balance allocation awareness.
Ensuring that replicas are spread out across multiple zones seems like
a good idea; can you elaborate a bit how you accomplish this?
Ensuring that replicas are spread out across multiple zones seems like
a good idea; can you elaborate a bit how you accomplish this?
It was stupidly easy with zen.
Count the number of zones you have. Subtract 1. That's the number of replicas you want.
Setup something on the host to put AWS_ZONE in an environment variable. Like something in rc.init. Our Ops guys did this when I asked, looks like its making an ec2 API call and pulling the zone out of the answer.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.