Currently, it appears that version numbers are not always taken into
account when resolving differences between nodes. It seems that the master
takes priority. Is this by design? It seems that, in principle, if updates
are being sent with external version numbers, it doesn’t matter about
masters or split brains etc because you can always work out which version
of a document is the correct one.
However, elasticsearch doesn’t keep a record of the version of deletes. So
when resolving differences between nodes it doesn’t know if a node simply
hasn’t received an update yet, or if it has actually deleted it since the
most recent update.
It’s easy to try out:
- Create an index with the default of 1 replica per primary shard,
spread across two nodes, then shutdown both nodes. - Start node 1.
- Send a document with external version 1.
- Shutdown node 1.
- Startup node 2.
- Startup node 1.
In step 5 node 2 becomes the master, so when node 1 starts, node 1 removes
the document you’ve just sent to it, because the master didn’t have it. If
elasticsearch kept records of deletes, it would have known that the reason
it didn’t exist on node 2 was because it hasn’t received it yet (rather
than because it had deleted it).
And there’s a variation on the above which enables a delete to be ‘undone’,
despite the fact that the version of the delete is greater than the version
of the insert.
I feel like external versioning provides a way for elasticsearch to always
know which version of a document is the most recent (whether it’s a delete
or an update). So, I feel like I shouldn’t really have to worry about which
order I start up my nodes.
So, I have two questions:
- Am I wrong/right/crazy?
- The current functionality of elasticsearch appears not to keep a
persistent record of deletes and therefore can't use that record in
resolving differences. Is there any reason why it couldn't? Are there any
architectural reasons stopping the behaviour being changed?
Cheers,
Tim.
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.