I'm considering using elasticsearch as a repository for a PoC I'm currently
This PoC models an application that needs durability but not isolability,
so I'm fine with the eventual consistency of reads against the most recent
As durability is paramount (we can't affort to lose the data unless 100% of
the nodes die) I've been exploring the option of setting every shard to
have N replicas where N is the number of nodes in the cluster.
From what I've read so far it is possible to dynamically set the number or
replicas which triggers a replication throttled replication process.
I would like to have some help on the following steps (I'm running ES in
embedded mode in a Java application):
1 - How can I set the number or replicas using the native Java client ?
2 - What happens if a node dies and the number of replicas is lowered to
the number of surviving ones?
3 - Is it possible, from a participating node, to access the list of nodes
in the cluster so I can use their count to set the number of replicas (step
4 - is it possible to hook a callback to the event of a node joining or
leaving the cluster ?
I envisioning the following mechanism:
a) - Start with one node, a given number of shards and 1 replica
b)- Each time a node joins I adjust the number or replicas to match the new
node count. In this case, there would be 2 replicas
c) - An arbitrary number of nodes might be added and I'd execute step b)
d) - At any time a node might leave the cluster and thus I need to lower
the number or replicas to the new node count (I assume that the cluster
would go ahead and proceed to compensate the lost replica by asking an
existing node to hold 2 replicas instead of one; is this stopped by
lowering the number or replicas?)
The ultimate goal is to make sure no data is loss unless 100% of the nodes
die before a new one can acquire a full replica.
Is this doable? Does this make sense at all ?
For the time being, I'm not worried about lack of disk space or bandwidth
as I'm still in the very early days of the PoC.
Thank you very much for all your work and help.
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to firstname.lastname@example.org.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/276418fa-812f-4af5-94a0-7362f5ba7931%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.