Why not reconcile shards/docs after split brain recovery?

Yes split brains are bad and we can avoid it by setting discovery.zen.minimum_master_nodes to (n/2+1) quorums. This question is to understand why Elasticsearch cannot attempt to reconcile/replay the diverged data when a cluster recovers back from split brain.

Yes, in case some documents have conflicting data, I understand there is data corruption but in many scenarios where documents on either halves have no conflicts, does Elasticsearch attempt to merge and reconcile the shards on the diverged halves?

If not that would be a cool feature to add but I am guessing this is not possible today. So I would love to understand more on what exactly happens when the cluster is coming out of a split brain and if I am right, why does Elasticsearch currently just discard all state from the half that was restarted (to join the other half)

How do you determine which documents take precedence?
How do you inform the user that they system is discarding data in favour of another?

In conflicting scenarios, I agree - you dont resolve conflicts and keep behavior as is (there is data loss here so i would even prefer calling this red when reconcilation attempt failed)

In non conflicting scenarios, which will be plenty does Elasticsearch try to automatically merge the diverged shards?

  • Example 1 - versioned docs where versioning is provided externally)
  • Example 2 - append only logstash/beat based log files which are append only and different log entries went to different halves

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.