Java client usage in the presence of split-brain


(magnus) #1

Hi,

Given that I have a cluster with 4 nodes, currently experiencing a
split-brain scenario so I have a fully functional cluster with 3 green
nodes, a sick cluster with 1 red node.
A java-client that talks to these 4 nodes (unicast zen discovery) will ~25%
of the time query the sick node, and get an error back. I'd expect it to
retry one of the healthy nodes, not just give up after an erroneous reply.

Do I need to handle this myself, or is this something the java client
should handle for me? I know there's some sort of retry logic in the
transport layer..

ES 0.90.7.

--
Magnus Haug

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/5dca4946-c26f-42f3-98ac-53bc2b478fc3%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Jörg Prante) #2

You have in all case do anything to prevent split brains, by setting
minimum master nodes, at least to 3.

Regarding the Java client, you can use the TransportClient with better
control of what nodes you want to connect to, but that will not solve your
problems. With minimum master node 3, you could attach the TransportClient
to these 3 master eligible nodes. More, you could use "sniff mode" where
every 5 secs the client can add/remove data nodes automatically. But, a
TransportClient can not detect split brains, so in most circumstances, you
will mess your data. Also, the retry logic is not related to cover split
brain issues.

Jörg

On Tue, Mar 18, 2014 at 10:04 AM, magnus@stack.no wrote:

Hi,

Given that I have a cluster with 4 nodes, currently experiencing a
split-brain scenario so I have a fully functional cluster with 3 green
nodes, a sick cluster with 1 red node.
A java-client that talks to these 4 nodes (unicast zen discovery) will
~25% of the time query the sick node, and get an error back. I'd expect it
to retry one of the healthy nodes, not just give up after an erroneous
reply.

Do I need to handle this myself, or is this something the java client
should handle for me? I know there's some sort of retry logic in the
transport layer..

ES 0.90.7.

--
Magnus Haug

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/5dca4946-c26f-42f3-98ac-53bc2b478fc3%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/5dca4946-c26f-42f3-98ac-53bc2b478fc3%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoE185cSMnT_k3BJFTFNGrAmrxkpApwkSCRiym2%2BhUbXXQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(system) #3