Elasticsearch cluster with AWS Spot Instances

(Felipe Hummel) #1

Hey guys, I've been thinking about the idea of running a elasticsearch
cluster in AWS/EC2 mixing "normal" (on-demand or reserved) instances with
spot instances.

The motivation is simple: spot instances run just as fast as a normal
instance but with a much lower price/hour. With the "only" caveat being
that the instance can die at any moment (if the floating price goes higher
than your bid).

The idea is to have a 50/50 or 30/70 split between normal and spot
instances. For example, 4 normal instances and 8 spot instances.

My questions/doubts are:

  • Has any one tried this kind of setup?
  • One "safety" issue is that master/primary shards can be allocated in spot
    instances which are prone to die. Anyway around this? Maybe allocate all
    primaries in the "normal instances"?
  • The number of spot instances can't be much higher than the number of
    normal instances. That would mean that, maybe, a shard woulnd't have a
    replica in at least one of the normal instances, which could be risky.
    Anyway around this?

Any thoughts?


Felipe Hummel

You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/20674cb7-5359-477f-b5c0-b2ac9b08cfce%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

(system) #2