Problem of sync elasticsearch data after failover


(fclddcn) #1

Hi everyone, I am new to elasticsearch. I'v setted up a small cluster (with
version 0.90.1).
Within the cluster I have 4 nodes total:

loadblancer node 1: on app1 server
loadblancer node 2: on app2 server
master node 1: stand alone server 1
master node 2: stand alone server 2

This architecture is come from the idea in this discussion:


http://stackoverflow.com/questions/17596182/connecting-to-elasticsearch-cluster-in-nest
See Duc.Duong's reply

each master node have config as:
node.master: true
node.data: true
discovery.zen.ping.multicast.enabled: false
discovery.zen.ping.unicast.hosts:
[":9300",":9300"]

I can get cluster have green status now. For me, the most important thing is
failover, and now this is basicly avaliable.
That is to say, I can shut down one master (using command "service
elasticsearch stop") and ES still work correctly, and when I start that
master again,
it joins the cluster as a new candidate master, the status goes green, and
data gets synced.

But,in extreme situation, if I did not start that master again, instead:
I shutdown the remaining master after inserting some documents, then,
I start the 2 masters, status goes green again, but the recently inserted
documents won't be synced.

How can I get these documents synced?
I'll thanks for any ideas.

--
View this message in context: http://elasticsearch-users.115913.n3.nabble.com/Problem-of-sync-elasticsearch-data-after-failover-tp4055275.html
Sent from the ElasticSearch Users mailing list archive at Nabble.com.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/1399166753095-4055275.post%40n3.nabble.com.
For more options, visit https://groups.google.com/d/optout.


(Mark Walkom) #2

Didn't you already ask this in this exact same thread?

Regards,
Mark Walkom

Infrastructure Engineer
Campaign Monitor
email: markw@campaignmonitor.com
web: www.campaignmonitor.com

On 4 May 2014 11:25, fclddcn fclddcn@gmail.com wrote:

Hi everyone, I am new to elasticsearch. I'v setted up a small cluster (with
version 0.90.1).
Within the cluster I have 4 nodes total:

loadblancer node 1: on app1 server
loadblancer node 2: on app2 server
master node 1: stand alone server 1
master node 2: stand alone server 2

This architecture is come from the idea in this discussion:

http://stackoverflow.com/questions/17596182/connecting-to-elasticsearch-cluster-in-nest
<
http://stackoverflow.com/questions/17596182/connecting-to-elasticsearch-cluster-in-nest

See Duc.Duong's reply

each master node have config as:
node.master: true
node.data: true
discovery.zen.ping.multicast.enabled: false
discovery.zen.ping.unicast.hosts:
[":9300",":9300"]

I can get cluster have green status now. For me, the most important thing
is
failover, and now this is basicly avaliable.
That is to say, I can shut down one master (using command "service
elasticsearch stop") and ES still work correctly, and when I start that
master again,
it joins the cluster as a new candidate master, the status goes green, and
data gets synced.

But,in extreme situation, if I did not start that master again, instead:
I shutdown the remaining master after inserting some documents, then,
I start the 2 masters, status goes green again, but the recently inserted
documents won't be synced.

How can I get these documents synced?
I'll thanks for any ideas.

--
View this message in context:
http://elasticsearch-users.115913.n3.nabble.com/Problem-of-sync-elasticsearch-data-after-failover-tp4055275.html
Sent from the ElasticSearch Users mailing list archive at Nabble.com.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/1399166753095-4055275.post%40n3.nabble.com
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAEM624bgMTftOOPxYd5oGf%3DMPKpY3MGLX%3DDA8gwFFsJfPFwzuQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(David Pilato) #3

Sorry Mark.

I guess it's my fault. I was looking at Google groups SPAM list today and I did not see that this message was already sent.

I think he tried to send more than one message as he did not see it in the ML.

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 6 mai 2014 à 08:15, Mark Walkom markw@campaignmonitor.com a écrit :

Didn't you already ask this in this exact same thread?

Regards,
Mark Walkom

Infrastructure Engineer
Campaign Monitor
email: markw@campaignmonitor.com
web: www.campaignmonitor.com

On 4 May 2014 11:25, fclddcn fclddcn@gmail.com wrote:
Hi everyone, I am new to elasticsearch. I'v setted up a small cluster (with
version 0.90.1).
Within the cluster I have 4 nodes total:

loadblancer node 1: on app1 server
loadblancer node 2: on app2 server
master node 1: stand alone server 1
master node 2: stand alone server 2

This architecture is come from the idea in this discussion:
http://stackoverflow.com/questions/17596182/connecting-to-elasticsearch-cluster-in-nest
http://stackoverflow.com/questions/17596182/connecting-to-elasticsearch-cluster-in-nest
See Duc.Duong's reply

each master node have config as:
node.master: true
node.data: true
discovery.zen.ping.multicast.enabled: false
discovery.zen.ping.unicast.hosts:
[":9300",":9300"]

I can get cluster have green status now. For me, the most important thing is
failover, and now this is basicly avaliable.
That is to say, I can shut down one master (using command "service
elasticsearch stop") and ES still work correctly, and when I start that
master again,
it joins the cluster as a new candidate master, the status goes green, and
data gets synced.

But,in extreme situation, if I did not start that master again, instead:
I shutdown the remaining master after inserting some documents, then,
I start the 2 masters, status goes green again, but the recently inserted
documents won't be synced.

How can I get these documents synced?
I'll thanks for any ideas.

--
View this message in context: http://elasticsearch-users.115913.n3.nabble.com/Problem-of-sync-elasticsearch-data-after-failover-tp4055275.html
Sent from the ElasticSearch Users mailing list archive at Nabble.com.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/1399166753095-4055275.post%40n3.nabble.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAEM624bgMTftOOPxYd5oGf%3DMPKpY3MGLX%3DDA8gwFFsJfPFwzuQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/BBD9D042-9282-4E42-9C6C-FFD766F888D1%40pilato.fr.
For more options, visit https://groups.google.com/d/optout.


(Mark Walkom) #4

Ahh! Oh well :slight_smile:

Regards,
Mark Walkom

Infrastructure Engineer
Campaign Monitor
email: markw@campaignmonitor.com
web: www.campaignmonitor.com

On 6 May 2014 16:26, David Pilato david@pilato.fr wrote:

Sorry Mark.

I guess it's my fault. I was looking at Google groups SPAM list today and
I did not see that this message was already sent.

I think he tried to send more than one message as he did not see it in the
ML.

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 6 mai 2014 à 08:15, Mark Walkom markw@campaignmonitor.com a écrit :

Didn't you already ask this in this exact same thread?

Regards,
Mark Walkom

Infrastructure Engineer
Campaign Monitor
email: markw@campaignmonitor.com
web: www.campaignmonitor.com

On 4 May 2014 11:25, fclddcn fclddcn@gmail.com wrote:

Hi everyone, I am new to elasticsearch. I'v setted up a small cluster
(with
version 0.90.1).
Within the cluster I have 4 nodes total:

loadblancer node 1: on app1 server
loadblancer node 2: on app2 server
master node 1: stand alone server 1
master node 2: stand alone server 2

This architecture is come from the idea in this discussion:

http://stackoverflow.com/questions/17596182/connecting-to-elasticsearch-cluster-in-nest
<
http://stackoverflow.com/questions/17596182/connecting-to-elasticsearch-cluster-in-nest

See Duc.Duong's reply

each master node have config as:
node.master: true
node.data: true
discovery.zen.ping.multicast.enabled: false
discovery.zen.ping.unicast.hosts:
[":9300",":9300"]

I can get cluster have green status now. For me, the most important thing
is
failover, and now this is basicly avaliable.
That is to say, I can shut down one master (using command "service
elasticsearch stop") and ES still work correctly, and when I start that
master again,
it joins the cluster as a new candidate master, the status goes green, and
data gets synced.

But,in extreme situation, if I did not start that master again, instead:
I shutdown the remaining master after inserting some documents, then,
I start the 2 masters, status goes green again, but the recently inserted
documents won't be synced.

How can I get these documents synced?
I'll thanks for any ideas.

--
View this message in context:
http://elasticsearch-users.115913.n3.nabble.com/Problem-of-sync-elasticsearch-data-after-failover-tp4055275.html
Sent from the ElasticSearch Users mailing list archive at Nabble.com.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/1399166753095-4055275.post%40n3.nabble.com
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAEM624bgMTftOOPxYd5oGf%3DMPKpY3MGLX%3DDA8gwFFsJfPFwzuQ%40mail.gmail.comhttps://groups.google.com/d/msgid/elasticsearch/CAEM624bgMTftOOPxYd5oGf%3DMPKpY3MGLX%3DDA8gwFFsJfPFwzuQ%40mail.gmail.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/BBD9D042-9282-4E42-9C6C-FFD766F888D1%40pilato.frhttps://groups.google.com/d/msgid/elasticsearch/BBD9D042-9282-4E42-9C6C-FFD766F888D1%40pilato.fr?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAEM624Zi_HC4zeRUH-2U%2B0F9Yb1eNXRPjVyiqE4XJKysYsJxSg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(fclddcn) #5

I think he tried to send more than one message as he did not see it in
the ML.

You are right, and sorry for that :slight_smile:

Can you give me a hint of my first question?

When the 2 servers dropped out (not shutdown manually),
I'll don't know which dropped out earlier or which did later, and this
should be automaticly done, what can I do? or I am wrong?

Thank you

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/ea8467c0-c541-439b-b0c8-1a233795f637%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(system) #6