On Wed, Jun 16, 2010 at 9:30 PM, Shay Banon
shay.banon@elasticsearch.com wrote:
You can't avoid split brain, it will happen once you get network
partitioning,
Not sure about this statement.
While I'm far from being a distributed systems expert, you're probably
way more than me, I think the impossibility result pretty depends on
the system assumptions.
Clearly, it's impossible to avoid split brain in fully asynchronous
networks with lost messages.
But, while you're forced to keep the assumption above for partition
tolerant and available systems, ES doesn't aim at tolerating
partitions (as you stated in your previous mail), so you can actually
avoid split brains by employing quorum algorithms and tolerating only
a given number of "failures" (because consistency needs to be
preserved), blocking the eventually partitioned nodes until they get
reconnected.
You could also choose to embrace partition tolerance over
availability, and adopt the solution above but avoid to block
partitioned nodes and just assume a fail-stop for them, or, assign a
static "owner" to a given index/shard, so that if the owner fails/gets
partitioned all writes to its index/shard will be prohibited to keep
consistency.
If, on the other hand, clients got
partitioned with (c) as well, then they will continue to work with (c),
while other clients will work with (a) and (b).
This kind of solutions describes indeed an available and partition
tolerant system, because both ends of your partition stay available
and you're actually sacrificing consistency.
Once the network partition is resolved, then some sort of data resolution
needs to occur, either by discarding the small cluster, or by doing version
/ conflict resolution.
Which is very hard for indexed data. Moreover, if this needs to be
done manually, users may actually miss inconsistencies or mess up
things ...
I honestly thought ES provided some kind of algorithm to avoid
inconsistencies, it doesn't seem to be the case. I'm not saying it's
bad, just that it's different than I expected.
Thanks for your attention,
Sergio B.
--
Sergio Bossa
http://www.linkedin.com/in/sergiob