Elasticsearch cluster with AWS Spot Instances


(Felipe Hummel) #1

Hey guys, I've been thinking about the idea of running a elasticsearch
cluster in AWS/EC2 mixing "normal" (on-demand or reserved) instances with
spot instances.

The motivation is simple: spot instances run just as fast as a normal
instance but with a much lower price/hour. With the "only" caveat being
that the instance can die at any moment (if the floating price goes higher
than your bid).

The idea is to have a 50/50 or 30/70 split between normal and spot
instances. For example, 4 normal instances and 8 spot instances.

My questions/doubts are:

  • Has any one tried this kind of setup?
  • One "safety" issue is that master/primary shards can be allocated in spot
    instances which are prone to die. Anyway around this? Maybe allocate all
    primaries in the "normal instances"?
  • The number of spot instances can't be much higher than the number of
    normal instances. That would mean that, maybe, a shard woulnd't have a
    replica in at least one of the normal instances, which could be risky.
    Anyway around this?

Any thoughts?

Thanks

Felipe Hummel

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/20674cb7-5359-477f-b5c0-b2ac9b08cfce%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(system) #2