Cluster state yellow


(Mohit Anchlia) #1

Cluster state is showing yellow. What's the best way to find why it's in
yellow state?

It has been up and stable so far except when I looked today it shows state
turned into yellow.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAOT3TWpSSEVTpYA-DB8Hgfyo9WdR4bVRoKtQ9c_PC%2BFZ1aQiKg%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Mark Walkom) #2

Yellow means it's got unallocated shards.

Install something like elastichq or kopf and you should be able to see why.

Regards,
Mark Walkom

Infrastructure Engineer
Campaign Monitor
email: markw@campaignmonitor.com
web: www.campaignmonitor.com

On 14 January 2014 08:30, Mohit Anchlia mohitanchlia@gmail.com wrote:

Cluster state is showing yellow. What's the best way to find why it's in
yellow state?

It has been up and stable so far except when I looked today it shows state
turned into yellow.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAOT3TWpSSEVTpYA-DB8Hgfyo9WdR4bVRoKtQ9c_PC%2BFZ1aQiKg%40mail.gmail.com
.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAEM624bfRfYA%2BitwKWN6oZj7qPss5xkS2xrm6Y-cqnvis3-dig%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.


(David Pilato) #3

Yellow means that some replicas are not allocated.
If you have only one node and have index set with one replica (default), yellow is the expected state.

--
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet | @elasticsearchfr

Le 13 janvier 2014 at 22:30:23, Mohit Anchlia (mohitanchlia@gmail.com) a écrit:

Cluster state is showing yellow. What's the best way to find why it's in yellow state?

It has been up and stable so far except when I looked today it shows state turned into yellow.

You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAOT3TWpSSEVTpYA-DB8Hgfyo9WdR4bVRoKtQ9c_PC%2BFZ1aQiKg%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/etPan.52d45be9.7c3dbd3d.376c%40MacBook-Air-de-David.local.
For more options, visit https://groups.google.com/groups/opt_out.


(Mohit Anchlia) #4

Thanks! It looks like one of the nodes even though it was up still it went
out of cluster. Is there no retry?

[2014-01-02 17:55:01,148][INFO ][cluster.service ] [10.80.140.14] master
{new [10.80.140.14][267foffDRhO6zzXlHg3YKA][inet[/10.80.140.14:9300]]{group=group1},
previous [10.80.140.59][YDHawZWnRUKukXDIIT5MJQ][inet[/10.80.140.59:9300]]{group=group2}},
removed {[10.80.140.59][YDHawZWnRUKukXDIIT5MJQ][inet[/10.80.140.59:9300]]{group=group2},},
reason: zen-disco-master_failed
([10.80.140.59][YDHawZWnRUKukXDIIT5MJQ][inet[/10.80.140.59:9300
]]{group=group2})

[2014-01-02 17:55:02,158][INFO ][cluster.service ] [10.80.140.14] removed
{[10.80.140.13][6cf39fcASKSn6dNNsSXguA][inet[/10.80.140.13:9300]]{group=group2},},
reason:
zen-disco-node_failed([10.80.140.13][6cf39fcASKSn6dNNsSXguA][inet[/10.80.140.13:9300]]{group=group2}),
reason failed to ping, tried [3] times, each with maximum [30s] timeout

On Mon, Jan 13, 2014 at 1:34 PM, David Pilato david@pilato.fr wrote:

Yellow means that some replicas are not allocated.
If you have only one node and have index set with one replica (default),
yellow is the expected state.

--
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet https://twitter.com/dadoonet | @elasticsearchfrhttps://twitter.com/elasticsearchfr

Le 13 janvier 2014 at 22:30:23, Mohit Anchlia (mohitanchlia@gmail.com//mohitanchlia@gmail.com)
a écrit:

Cluster state is showing yellow. What's the best way to find why it's in
yellow state?

It has been up and stable so far except when I looked today it shows state
turned into yellow.

You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAOT3TWpSSEVTpYA-DB8Hgfyo9WdR4bVRoKtQ9c_PC%2BFZ1aQiKg%40mail.gmail.com
.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/etPan.52d45be9.7c3dbd3d.376c%40MacBook-Air-de-David.local
.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAOT3TWpLn_AgMKcTwQuGD-CHHe6ptg%3DRxaNGLPxFn3kroed%3DVQ%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.


(David Pilato) #5

It sounds like a split brain issue or something.

How many nodes do you have?
Did you set minimum master node?

--
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet | @elasticsearchfr

Le 13 janvier 2014 at 22:48:32, Mohit Anchlia (mohitanchlia@gmail.com) a écrit:

Thanks! It looks like one of the nodes even though it was up still it went out of cluster. Is there no retry?

[2014-01-02 17:55:01,148][INFO ][cluster.service ] [10.80.140.14] master {new [10.80.140.14][267foffDRhO6zzXlHg3YKA][inet[/10.80.140.14:9300]]{group=group1}, previous [10.80.140.59][YDHawZWnRUKukXDIIT5MJQ][inet[/10.80.140.59:9300]]{group=group2}}, removed {[10.80.140.59][YDHawZWnRUKukXDIIT5MJQ][inet[/10.80.140.59:9300]]{group=group2},}, reason: zen-disco-master_failed ([10.80.140.59][YDHawZWnRUKukXDIIT5MJQ][inet[/10.80.140.59:9300]]{group=group2})

[2014-01-02 17:55:02,158][INFO ][cluster.service ] [10.80.140.14] removed {[10.80.140.13][6cf39fcASKSn6dNNsSXguA][inet[/10.80.140.13:9300]]{group=group2},}, reason: zen-disco-node_failed([10.80.140.13][6cf39fcASKSn6dNNsSXguA][inet[/10.80.140.13:9300]]{group=group2}), reason failed to ping, tried [3] times, each with maximum [30s] timeout

On Mon, Jan 13, 2014 at 1:34 PM, David Pilato david@pilato.fr wrote:
Yellow means that some replicas are not allocated.
If you have only one node and have index set with one replica (default), yellow is the expected state.

--
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet | @elasticsearchfr

Le 13 janvier 2014 at 22:30:23, Mohit Anchlia (mohitanchlia@gmail.com) a écrit:

Cluster state is showing yellow. What's the best way to find why it's in yellow state?

It has been up and stable so far except when I looked today it shows state turned into yellow.

You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAOT3TWpSSEVTpYA-DB8Hgfyo9WdR4bVRoKtQ9c_PC%2BFZ1aQiKg%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/etPan.52d45be9.7c3dbd3d.376c%40MacBook-Air-de-David.local.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAOT3TWpLn_AgMKcTwQuGD-CHHe6ptg%3DRxaNGLPxFn3kroed%3DVQ%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/etPan.52d46968.5577f8e1.376c%40MacBook-Air-de-David.local.
For more options, visit https://groups.google.com/groups/opt_out.


(Mohit Anchlia) #6

I have 5 nodes. I am using all the default settings.

On Mon, Jan 13, 2014 at 2:32 PM, David Pilato david@pilato.fr wrote:

It sounds like a split brain issue or something.

How many nodes do you have?
Did you set minimum master node?

--
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet https://twitter.com/dadoonet | @elasticsearchfrhttps://twitter.com/elasticsearchfr

Le 13 janvier 2014 at 22:48:32, Mohit Anchlia (mohitanchlia@gmail.com//mohitanchlia@gmail.com)
a écrit:

Thanks! It looks like one of the nodes even though it was up still it went
out of cluster. Is there no retry?

[2014-01-02 17:55:01,148][INFO ][cluster.service ] [10.80.140.14] master
{new [10.80.140.14][267foffDRhO6zzXlHg3YKA][inet[/10.80.140.14:9300]]{group=group1},
previous [10.80.140.59][YDHawZWnRUKukXDIIT5MJQ][inet[/10.80.140.59:9300]]{group=group2}},
removed {[10.80.140.59][YDHawZWnRUKukXDIIT5MJQ][inet[/10.80.140.59:9300]]{group=group2},},
reason: zen-disco-master_failed
([10.80.140.59][YDHawZWnRUKukXDIIT5MJQ][inet[/10.80.140.59:9300
]]{group=group2})

[2014-01-02 17:55:02,158][INFO ][cluster.service ] [10.80.140.14] removed
{[10.80.140.13][6cf39fcASKSn6dNNsSXguA][inet[/10.80.140.13:9300]]{group=group2},},
reason:
zen-disco-node_failed([10.80.140.13][6cf39fcASKSn6dNNsSXguA][inet[/10.80.140.13:9300]]{group=group2}),
reason failed to ping, tried [3] times, each with maximum [30s] timeout

On Mon, Jan 13, 2014 at 1:34 PM, David Pilato david@pilato.fr wrote:

Yellow means that some replicas are not allocated.
If you have only one node and have index set with one replica (default),
yellow is the expected state.

 --

David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet https://twitter.com/dadoonet | @elasticsearchfrhttps://twitter.com/elasticsearchfr

Le 13 janvier 2014 at 22:30:23, Mohit Anchlia (mohitanchlia@gmail.com//mohitanchlia@gmail.com)
a écrit:

Cluster state is showing yellow. What's the best way to find why it's
in yellow state?

It has been up and stable so far except when I looked today it shows
state turned into yellow.

You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAOT3TWpSSEVTpYA-DB8Hgfyo9WdR4bVRoKtQ9c_PC%2BFZ1aQiKg%40mail.gmail.com
.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/etPan.52d45be9.7c3dbd3d.376c%40MacBook-Air-de-David.local.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAOT3TWpLn_AgMKcTwQuGD-CHHe6ptg%3DRxaNGLPxFn3kroed%3DVQ%40mail.gmail.com
.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/etPan.52d46968.5577f8e1.376c%40MacBook-Air-de-David.local
.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAOT3TWrsU-k9u6hir2CWoobhnaOTD_3Bjk3Xf_ZjMDSBkB0DeQ%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.


(David Pilato) #7

So I think you should first set minimum master node to 3. See http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/cluster-update-settings.html#_discovery

Probably you need to close your isolated node and restart it.

--
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet | @elasticsearchfr

Le 13 janvier 2014 at 23:58:33, Mohit Anchlia (mohitanchlia@gmail.com) a écrit:

I have 5 nodes. I am using all the default settings.

On Mon, Jan 13, 2014 at 2:32 PM, David Pilato david@pilato.fr wrote:
It sounds like a split brain issue or something.

How many nodes do you have?
Did you set minimum master node?

--
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet | @elasticsearchfr

Le 13 janvier 2014 at 22:48:32, Mohit Anchlia (mohitanchlia@gmail.com) a écrit:

Thanks! It looks like one of the nodes even though it was up still it went out of cluster. Is there no retry?

[2014-01-02 17:55:01,148][INFO ][cluster.service ] [10.80.140.14] master {new [10.80.140.14][267foffDRhO6zzXlHg3YKA][inet[/10.80.140.14:9300]]{group=group1}, previous [10.80.140.59][YDHawZWnRUKukXDIIT5MJQ][inet[/10.80.140.59:9300]]{group=group2}}, removed {[10.80.140.59][YDHawZWnRUKukXDIIT5MJQ][inet[/10.80.140.59:9300]]{group=group2},}, reason: zen-disco-master_failed ([10.80.140.59][YDHawZWnRUKukXDIIT5MJQ][inet[/10.80.140.59:9300]]{group=group2})

[2014-01-02 17:55:02,158][INFO ][cluster.service ] [10.80.140.14] removed {[10.80.140.13][6cf39fcASKSn6dNNsSXguA][inet[/10.80.140.13:9300]]{group=group2},}, reason: zen-disco-node_failed([10.80.140.13][6cf39fcASKSn6dNNsSXguA][inet[/10.80.140.13:9300]]{group=group2}), reason failed to ping, tried [3] times, each with maximum [30s] timeout

On Mon, Jan 13, 2014 at 1:34 PM, David Pilato david@pilato.fr wrote:
Yellow means that some replicas are not allocated.
If you have only one node and have index set with one replica (default), yellow is the expected state.

--
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet | @elasticsearchfr

Le 13 janvier 2014 at 22:30:23, Mohit Anchlia (mohitanchlia@gmail.com) a écrit:

Cluster state is showing yellow. What's the best way to find why it's in yellow state?

It has been up and stable so far except when I looked today it shows state turned into yellow.

You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAOT3TWpSSEVTpYA-DB8Hgfyo9WdR4bVRoKtQ9c_PC%2BFZ1aQiKg%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/etPan.52d45be9.7c3dbd3d.376c%40MacBook-Air-de-David.local.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAOT3TWpLn_AgMKcTwQuGD-CHHe6ptg%3DRxaNGLPxFn3kroed%3DVQ%40mail.gmail.com.

For more options, visit https://groups.google.com/groups/opt_out.

You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/etPan.52d46968.5577f8e1.376c%40MacBook-Air-de-David.local.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAOT3TWrsU-k9u6hir2CWoobhnaOTD_3Bjk3Xf_ZjMDSBkB0DeQ%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/etPan.52d47036.77465f01.376c%40MacBook-Air-de-David.local.
For more options, visit https://groups.google.com/groups/opt_out.


(Mohit Anchlia) #8

Is the default setting 3? How do I go about selecting right configuration
setting for this property?

On Mon, Jan 13, 2014 at 3:01 PM, David Pilato david@pilato.fr wrote:

So I think you should first set minimum master node to 3. See
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/cluster-update-settings.html#_discovery

Probably you need to close your isolated node and restart it.

--
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet https://twitter.com/dadoonet | @elasticsearchfrhttps://twitter.com/elasticsearchfr

Le 13 janvier 2014 at 23:58:33, Mohit Anchlia (mohitanchlia@gmail.com//mohitanchlia@gmail.com)
a écrit:

I have 5 nodes. I am using all the default settings.

On Mon, Jan 13, 2014 at 2:32 PM, David Pilato david@pilato.fr wrote:

It sounds like a split brain issue or something.

How many nodes do you have?
Did you set minimum master node?

 --

David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet https://twitter.com/dadoonet | @elasticsearchfrhttps://twitter.com/elasticsearchfr

Le 13 janvier 2014 at 22:48:32, Mohit Anchlia (mohitanchlia@gmail.com//mohitanchlia@gmail.com)
a écrit:

Thanks! It looks like one of the nodes even though it was up still it
went out of cluster. Is there no retry?

[2014-01-02 17:55:01,148][INFO ][cluster.service ] [10.80.140.14] master
{new [10.80.140.14][267foffDRhO6zzXlHg3YKA][inet[/10.80.140.14:9300]]{group=group1},
previous [10.80.140.59][YDHawZWnRUKukXDIIT5MJQ][inet[/10.80.140.59:9300]]{group=group2}},
removed {[10.80.140.59][YDHawZWnRUKukXDIIT5MJQ][inet[/10.80.140.59:9300]]{group=group2},},
reason: zen-disco-master_failed
([10.80.140.59][YDHawZWnRUKukXDIIT5MJQ][inet[/10.80.140.59:9300
]]{group=group2})

[2014-01-02 17:55:02,158][INFO ][cluster.service ] [10.80.140.14] removed
{[10.80.140.13][6cf39fcASKSn6dNNsSXguA][inet[/10.80.140.13:9300]]{group=group2},},
reason:
zen-disco-node_failed([10.80.140.13][6cf39fcASKSn6dNNsSXguA][inet[/10.80.140.13:9300]]{group=group2}),
reason failed to ping, tried [3] times, each with maximum [30s] timeout

On Mon, Jan 13, 2014 at 1:34 PM, David Pilato david@pilato.fr wrote:

Yellow means that some replicas are not allocated.
If you have only one node and have index set with one replica
(default), yellow is the expected state.

 --

David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet https://twitter.com/dadoonet | @elasticsearchfrhttps://twitter.com/elasticsearchfr

Le 13 janvier 2014 at 22:30:23, Mohit Anchlia (mohitanchlia@gmail.com//mohitanchlia@gmail.com)
a écrit:

Cluster state is showing yellow. What's the best way to find why it's
in yellow state?

It has been up and stable so far except when I looked today it shows
state turned into yellow.

You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAOT3TWpSSEVTpYA-DB8Hgfyo9WdR4bVRoKtQ9c_PC%2BFZ1aQiKg%40mail.gmail.com
.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/etPan.52d45be9.7c3dbd3d.376c%40MacBook-Air-de-David.local.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAOT3TWpLn_AgMKcTwQuGD-CHHe6ptg%3DRxaNGLPxFn3kroed%3DVQ%40mail.gmail.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/etPan.52d46968.5577f8e1.376c%40MacBook-Air-de-David.local.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAOT3TWrsU-k9u6hir2CWoobhnaOTD_3Bjk3Xf_ZjMDSBkB0DeQ%40mail.gmail.com
.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/etPan.52d47036.77465f01.376c%40MacBook-Air-de-David.local
.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAOT3TWqbrUh5p2t74Bc_6kL%3DxjAcLGrJzJ6wt_fRQqy6Dj-DKg%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.


(David Pilato) #9

No it's not the default setting.
min master node = total num of nodes / 2 + 1

You have 5 nodes, 5 / 2 + 1 = 3

--
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet | @elasticsearchfr

Le 14 janvier 2014 at 00:06:34, Mohit Anchlia (mohitanchlia@gmail.com) a écrit:

Is the default setting 3? How do I go about selecting right configuration setting for this property?

On Mon, Jan 13, 2014 at 3:01 PM, David Pilato david@pilato.fr wrote:
So I think you should first set minimum master node to 3. See http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/cluster-update-settings.html#_discovery

Probably you need to close your isolated node and restart it.

--
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet | @elasticsearchfr

Le 13 janvier 2014 at 23:58:33, Mohit Anchlia (mohitanchlia@gmail.com) a écrit:

I have 5 nodes. I am using all the default settings.

On Mon, Jan 13, 2014 at 2:32 PM, David Pilato david@pilato.fr wrote:
It sounds like a split brain issue or something.

How many nodes do you have?
Did you set minimum master node?

--
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet | @elasticsearchfr

Le 13 janvier 2014 at 22:48:32, Mohit Anchlia (mohitanchlia@gmail.com) a écrit:

Thanks! It looks like one of the nodes even though it was up still it went out of cluster. Is there no retry?

[2014-01-02 17:55:01,148][INFO ][cluster.service ] [10.80.140.14] master {new [10.80.140.14][267foffDRhO6zzXlHg3YKA][inet[/10.80.140.14:9300]]{group=group1}, previous [10.80.140.59][YDHawZWnRUKukXDIIT5MJQ][inet[/10.80.140.59:9300]]{group=group2}}, removed {[10.80.140.59][YDHawZWnRUKukXDIIT5MJQ][inet[/10.80.140.59:9300]]{group=group2},}, reason: zen-disco-master_failed ([10.80.140.59][YDHawZWnRUKukXDIIT5MJQ][inet[/10.80.140.59:9300]]{group=group2})

[2014-01-02 17:55:02,158][INFO ][cluster.service ] [10.80.140.14] removed {[10.80.140.13][6cf39fcASKSn6dNNsSXguA][inet[/10.80.140.13:9300]]{group=group2},}, reason: zen-disco-node_failed([10.80.140.13][6cf39fcASKSn6dNNsSXguA][inet[/10.80.140.13:9300]]{group=group2}), reason failed to ping, tried [3] times, each with maximum [30s] timeout

On Mon, Jan 13, 2014 at 1:34 PM, David Pilato david@pilato.fr wrote:
Yellow means that some replicas are not allocated.
If you have only one node and have index set with one replica (default), yellow is the expected state.

--
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet | @elasticsearchfr

Le 13 janvier 2014 at 22:30:23, Mohit Anchlia (mohitanchlia@gmail.com) a écrit:

Cluster state is showing yellow. What's the best way to find why it's in yellow state?

It has been up and stable so far except when I looked today it shows state turned into yellow.

You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAOT3TWpSSEVTpYA-DB8Hgfyo9WdR4bVRoKtQ9c_PC%2BFZ1aQiKg%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/etPan.52d45be9.7c3dbd3d.376c%40MacBook-Air-de-David.local.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAOT3TWpLn_AgMKcTwQuGD-CHHe6ptg%3DRxaNGLPxFn3kroed%3DVQ%40mail.gmail.com.

For more options, visit https://groups.google.com/groups/opt_out.

You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/etPan.52d46968.5577f8e1.376c%40MacBook-Air-de-David.local.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAOT3TWrsU-k9u6hir2CWoobhnaOTD_3Bjk3Xf_ZjMDSBkB0DeQ%40mail.gmail.com.

For more options, visit https://groups.google.com/groups/opt_out.

You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/etPan.52d47036.77465f01.376c%40MacBook-Air-de-David.local.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAOT3TWqbrUh5p2t74Bc_6kL%3DxjAcLGrJzJ6wt_fRQqy6Dj-DKg%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/etPan.52d47982.5e884adc.376c%40MacBook-Air-de-David.local.
For more options, visit https://groups.google.com/groups/opt_out.


(Mohit Anchlia) #10

I am trying to understand why I should change it? This setting looks like
the right setting in most cases. Did you see anything out of ordinary in
that log message I pasted?

On Mon, Jan 13, 2014 at 3:40 PM, David Pilato david@pilato.fr wrote:

No it's not the default setting.
min master node = total num of nodes / 2 + 1

You have 5 nodes, 5 / 2 + 1 = 3

--
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet https://twitter.com/dadoonet | @elasticsearchfrhttps://twitter.com/elasticsearchfr

Le 14 janvier 2014 at 00:06:34, Mohit Anchlia (mohitanchlia@gmail.com//mohitanchlia@gmail.com)
a écrit:

Is the default setting 3? How do I go about selecting right configuration
setting for this property?

On Mon, Jan 13, 2014 at 3:01 PM, David Pilato david@pilato.fr wrote:

So I think you should first set minimum master node to 3. See
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/cluster-update-settings.html#_discovery

Probably you need to close your isolated node and restart it.

 --

David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet https://twitter.com/dadoonet | @elasticsearchfrhttps://twitter.com/elasticsearchfr

Le 13 janvier 2014 at 23:58:33, Mohit Anchlia (mohitanchlia@gmail.com//mohitanchlia@gmail.com)
a écrit:

I have 5 nodes. I am using all the default settings.

On Mon, Jan 13, 2014 at 2:32 PM, David Pilato david@pilato.fr wrote:

It sounds like a split brain issue or something.

How many nodes do you have?
Did you set minimum master node?

 --

David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet https://twitter.com/dadoonet | @elasticsearchfrhttps://twitter.com/elasticsearchfr

Le 13 janvier 2014 at 22:48:32, Mohit Anchlia (mohitanchlia@gmail.com//mohitanchlia@gmail.com)
a écrit:

Thanks! It looks like one of the nodes even though it was up still it
went out of cluster. Is there no retry?

[2014-01-02 17:55:01,148][INFO ][cluster.service ] [10.80.140.14] master
{new [10.80.140.14][267foffDRhO6zzXlHg3YKA][inet[/10.80.140.14:9300]]{group=group1},
previous [10.80.140.59][YDHawZWnRUKukXDIIT5MJQ][inet[/10.80.140.59:9300]]{group=group2}},
removed {[10.80.140.59][YDHawZWnRUKukXDIIT5MJQ][inet[/10.80.140.59:9300]]{group=group2},},
reason: zen-disco-master_failed
([10.80.140.59][YDHawZWnRUKukXDIIT5MJQ][inet[/10.80.140.59:9300
]]{group=group2})

[2014-01-02 17:55:02,158][INFO ][cluster.service ] [10.80.140.14]
removed {[10.80.140.13][6cf39fcASKSn6dNNsSXguA][inet[/10.80.140.13:9300]]{group=group2},},
reason:
zen-disco-node_failed([10.80.140.13][6cf39fcASKSn6dNNsSXguA][inet[/10.80.140.13:9300]]{group=group2}),
reason failed to ping, tried [3] times, each with maximum [30s] timeout

On Mon, Jan 13, 2014 at 1:34 PM, David Pilato david@pilato.fr wrote:

Yellow means that some replicas are not allocated.
If you have only one node and have index set with one replica
(default), yellow is the expected state.

 --

David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet https://twitter.com/dadoonet | @elasticsearchfrhttps://twitter.com/elasticsearchfr

Le 13 janvier 2014 at 22:30:23, Mohit Anchlia (mohitanchlia@gmail.com//mohitanchlia@gmail.com)
a écrit:

Cluster state is showing yellow. What's the best way to find why
it's in yellow state?

It has been up and stable so far except when I looked today it shows
state turned into yellow.

You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAOT3TWpSSEVTpYA-DB8Hgfyo9WdR4bVRoKtQ9c_PC%2BFZ1aQiKg%40mail.gmail.com
.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/etPan.52d45be9.7c3dbd3d.376c%40MacBook-Air-de-David.local.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAOT3TWpLn_AgMKcTwQuGD-CHHe6ptg%3DRxaNGLPxFn3kroed%3DVQ%40mail.gmail.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/etPan.52d46968.5577f8e1.376c%40MacBook-Air-de-David.local.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAOT3TWrsU-k9u6hir2CWoobhnaOTD_3Bjk3Xf_ZjMDSBkB0DeQ%40mail.gmail.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/etPan.52d47036.77465f01.376c%40MacBook-Air-de-David.local.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAOT3TWqbrUh5p2t74Bc_6kL%3DxjAcLGrJzJ6wt_fRQqy6Dj-DKg%40mail.gmail.com
.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/etPan.52d47982.5e884adc.376c%40MacBook-Air-de-David.local
.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAOT3TWqx_KLF_p8nJ4m3On-nSvRncY9OdHa8KS1HDjcRHhfQhQ%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Mark Walkom) #11

The logs shows that a node was not contactable and so was removed from the
cluster.
That could be a load issue on the node, network issues, or something else.
Check the logs on the node that disconnected and see what you find.

If you leave the defaults, you risk the chance of a split-brain situation,
where nodes lose contact with each other and then decide to become a
master. This can cause you to end up with multiple masters in multiple
clusters, which is not good for data integrity.

Setting the min masters means that if a node cannot contact N masters, it
will refuse connections.

Regards,
Mark Walkom

Infrastructure Engineer
Campaign Monitor
email: markw@campaignmonitor.com
web: www.campaignmonitor.com

On 14 January 2014 10:46, Mohit Anchlia mohitanchlia@gmail.com wrote:

I am trying to understand why I should change it? This setting looks like
the right setting in most cases. Did you see anything out of ordinary in
that log message I pasted?

On Mon, Jan 13, 2014 at 3:40 PM, David Pilato david@pilato.fr wrote:

No it's not the default setting.
min master node = total num of nodes / 2 + 1

You have 5 nodes, 5 / 2 + 1 = 3

--
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet https://twitter.com/dadoonet | @elasticsearchfrhttps://twitter.com/elasticsearchfr

Le 14 janvier 2014 at 00:06:34, Mohit Anchlia (mohitanchlia@gmail.com//mohitanchlia@gmail.com)
a écrit:

Is the default setting 3? How do I go about selecting right configuration
setting for this property?

On Mon, Jan 13, 2014 at 3:01 PM, David Pilato david@pilato.fr wrote:

So I think you should first set minimum master node to 3. See
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/cluster-update-settings.html#_discovery

Probably you need to close your isolated node and restart it.

 --

David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet https://twitter.com/dadoonet | @elasticsearchfrhttps://twitter.com/elasticsearchfr

Le 13 janvier 2014 at 23:58:33, Mohit Anchlia (mohitanchlia@gmail.com//mohitanchlia@gmail.com)
a écrit:

I have 5 nodes. I am using all the default settings.

On Mon, Jan 13, 2014 at 2:32 PM, David Pilato david@pilato.fr wrote:

It sounds like a split brain issue or something.

How many nodes do you have?
Did you set minimum master node?

 --

David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet https://twitter.com/dadoonet | @elasticsearchfrhttps://twitter.com/elasticsearchfr

Le 13 janvier 2014 at 22:48:32, Mohit Anchlia (mohitanchlia@gmail.com//mohitanchlia@gmail.com)
a écrit:

Thanks! It looks like one of the nodes even though it was up still
it went out of cluster. Is there no retry?

[2014-01-02 17:55:01,148][INFO ][cluster.service ] [10.80.140.14]
master {new [10.80.140.14][267foffDRhO6zzXlHg3YKA][inet[/10.80.140.14:9300]]{group=group1},
previous [10.80.140.59][YDHawZWnRUKukXDIIT5MJQ][inet[/10.80.140.59:9300]]{group=group2}},
removed {[10.80.140.59][YDHawZWnRUKukXDIIT5MJQ][inet[/10.80.140.59:9300]]{group=group2},},
reason: zen-disco-master_failed
([10.80.140.59][YDHawZWnRUKukXDIIT5MJQ][inet[/10.80.140.59:9300
]]{group=group2})

[2014-01-02 17:55:02,158][INFO ][cluster.service ] [10.80.140.14]
removed {[10.80.140.13][6cf39fcASKSn6dNNsSXguA][inet[/10.80.140.13:9300]]{group=group2},},
reason:
zen-disco-node_failed([10.80.140.13][6cf39fcASKSn6dNNsSXguA][inet[/10.80.140.13:9300]]{group=group2}),
reason failed to ping, tried [3] times, each with maximum [30s] timeout

On Mon, Jan 13, 2014 at 1:34 PM, David Pilato david@pilato.fr wrote:

Yellow means that some replicas are not allocated.
If you have only one node and have index set with one replica
(default), yellow is the expected state.

 --

David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet https://twitter.com/dadoonet | @elasticsearchfrhttps://twitter.com/elasticsearchfr

Le 13 janvier 2014 at 22:30:23, Mohit Anchlia (mohitanchlia@gmail.com//mohitanchlia@gmail.com)
a écrit:

Cluster state is showing yellow. What's the best way to find why
it's in yellow state?

It has been up and stable so far except when I looked today it shows
state turned into yellow.

You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAOT3TWpSSEVTpYA-DB8Hgfyo9WdR4bVRoKtQ9c_PC%2BFZ1aQiKg%40mail.gmail.com
.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/etPan.52d45be9.7c3dbd3d.376c%40MacBook-Air-de-David.local.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAOT3TWpLn_AgMKcTwQuGD-CHHe6ptg%3DRxaNGLPxFn3kroed%3DVQ%40mail.gmail.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/etPan.52d46968.5577f8e1.376c%40MacBook-Air-de-David.local.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAOT3TWrsU-k9u6hir2CWoobhnaOTD_3Bjk3Xf_ZjMDSBkB0DeQ%40mail.gmail.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/etPan.52d47036.77465f01.376c%40MacBook-Air-de-David.local.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAOT3TWqbrUh5p2t74Bc_6kL%3DxjAcLGrJzJ6wt_fRQqy6Dj-DKg%40mail.gmail.com
.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/etPan.52d47982.5e884adc.376c%40MacBook-Air-de-David.local
.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAOT3TWqx_KLF_p8nJ4m3On-nSvRncY9OdHa8KS1HDjcRHhfQhQ%40mail.gmail.com
.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAEM624aGEw8HqnMP--Fb-U%2BCjB58VBFb1bQenk_RmenNKAmMHg%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Mohit Anchlia) #12

How would the nodes on the other side of the network become master with the
formula (total num of nodes / 2 + 1) one side will always become
inaccessible?

On Mon, Jan 13, 2014 at 3:50 PM, Mark Walkom markw@campaignmonitor.comwrote:

The logs shows that a node was not contactable and so was removed from the
cluster.
That could be a load issue on the node, network issues, or something else.
Check the logs on the node that disconnected and see what you find.

If you leave the defaults, you risk the chance of a split-brain situation,
where nodes lose contact with each other and then decide to become a
master. This can cause you to end up with multiple masters in multiple
clusters, which is not good for data integrity.

Setting the min masters means that if a node cannot contact N masters, it
will refuse connections.

Regards,
Mark Walkom

Infrastructure Engineer
Campaign Monitor
email: markw@campaignmonitor.com
web: www.campaignmonitor.com

On 14 January 2014 10:46, Mohit Anchlia mohitanchlia@gmail.com wrote:

I am trying to understand why I should change it? This setting looks like
the right setting in most cases. Did you see anything out of ordinary in
that log message I pasted?

On Mon, Jan 13, 2014 at 3:40 PM, David Pilato david@pilato.fr wrote:

No it's not the default setting.
min master node = total num of nodes / 2 + 1

You have 5 nodes, 5 / 2 + 1 = 3

--
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet https://twitter.com/dadoonet | @elasticsearchfrhttps://twitter.com/elasticsearchfr

Le 14 janvier 2014 at 00:06:34, Mohit Anchlia (mohitanchlia@gmail.com//mohitanchlia@gmail.com)
a écrit:

Is the default setting 3? How do I go about selecting right
configuration setting for this property?

On Mon, Jan 13, 2014 at 3:01 PM, David Pilato david@pilato.fr wrote:

So I think you should first set minimum master node to 3. See
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/cluster-update-settings.html#_discovery

Probably you need to close your isolated node and restart it.

 --

David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet https://twitter.com/dadoonet | @elasticsearchfrhttps://twitter.com/elasticsearchfr

Le 13 janvier 2014 at 23:58:33, Mohit Anchlia (mohitanchlia@gmail.com//mohitanchlia@gmail.com)
a écrit:

I have 5 nodes. I am using all the default settings.

On Mon, Jan 13, 2014 at 2:32 PM, David Pilato david@pilato.fr wrote:

It sounds like a split brain issue or something.

How many nodes do you have?
Did you set minimum master node?

 --

David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet https://twitter.com/dadoonet | @elasticsearchfrhttps://twitter.com/elasticsearchfr

Le 13 janvier 2014 at 22:48:32, Mohit Anchlia (mohitanchlia@gmail.com//mohitanchlia@gmail.com)
a écrit:

Thanks! It looks like one of the nodes even though it was up still
it went out of cluster. Is there no retry?

[2014-01-02 17:55:01,148][INFO ][cluster.service ] [10.80.140.14]
master {new [10.80.140.14][267foffDRhO6zzXlHg3YKA][inet[/10.80.140.14:9300]]{group=group1},
previous [10.80.140.59][YDHawZWnRUKukXDIIT5MJQ][inet[/10.80.140.59:9300]]{group=group2}},
removed {[10.80.140.59][YDHawZWnRUKukXDIIT5MJQ][inet[/10.80.140.59:9300]]{group=group2},},
reason: zen-disco-master_failed
([10.80.140.59][YDHawZWnRUKukXDIIT5MJQ][inet[/10.80.140.59:9300
]]{group=group2})

[2014-01-02 17:55:02,158][INFO ][cluster.service ] [10.80.140.14]
removed {[10.80.140.13][6cf39fcASKSn6dNNsSXguA][inet[/10.80.140.13:9300]]{group=group2},},
reason:
zen-disco-node_failed([10.80.140.13][6cf39fcASKSn6dNNsSXguA][inet[/10.80.140.13:9300]]{group=group2}),
reason failed to ping, tried [3] times, each with maximum [30s] timeout

On Mon, Jan 13, 2014 at 1:34 PM, David Pilato david@pilato.fr wrote:

Yellow means that some replicas are not allocated.
If you have only one node and have index set with one replica
(default), yellow is the expected state.

 --

David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet https://twitter.com/dadoonet | @elasticsearchfrhttps://twitter.com/elasticsearchfr

Le 13 janvier 2014 at 22:30:23, Mohit Anchlia (mohitanchlia@gmail.com//mohitanchlia@gmail.com)
a écrit:

Cluster state is showing yellow. What's the best way to find why
it's in yellow state?

It has been up and stable so far except when I looked today it shows
state turned into yellow.

You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAOT3TWpSSEVTpYA-DB8Hgfyo9WdR4bVRoKtQ9c_PC%2BFZ1aQiKg%40mail.gmail.com
.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/etPan.52d45be9.7c3dbd3d.376c%40MacBook-Air-de-David.local.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAOT3TWpLn_AgMKcTwQuGD-CHHe6ptg%3DRxaNGLPxFn3kroed%3DVQ%40mail.gmail.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/etPan.52d46968.5577f8e1.376c%40MacBook-Air-de-David.local.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAOT3TWrsU-k9u6hir2CWoobhnaOTD_3Bjk3Xf_ZjMDSBkB0DeQ%40mail.gmail.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/etPan.52d47036.77465f01.376c%40MacBook-Air-de-David.local.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAOT3TWqbrUh5p2t74Bc_6kL%3DxjAcLGrJzJ6wt_fRQqy6Dj-DKg%40mail.gmail.com
.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/etPan.52d47982.5e884adc.376c%40MacBook-Air-de-David.local
.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAOT3TWqx_KLF_p8nJ4m3On-nSvRncY9OdHa8KS1HDjcRHhfQhQ%40mail.gmail.com
.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAEM624aGEw8HqnMP--Fb-U%2BCjB58VBFb1bQenk_RmenNKAmMHg%40mail.gmail.com
.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAOT3TWrGJOS4%2BA6bm%3DKLm7aTDoQxX8TZGM7CgxVoGAeO%3D-XNLA%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Mark Walkom) #13

That depends on however many nodes lose contact, but it's a possibility
that if you lose 2 of your nodes at once they will just stop responding and
the other 3 will be ok.
However you maintain cluster integrity.

Regards,
Mark Walkom

Infrastructure Engineer
Campaign Monitor
email: markw@campaignmonitor.com
web: www.campaignmonitor.com

On 14 January 2014 10:55, Mohit Anchlia mohitanchlia@gmail.com wrote:

How would the nodes on the other side of the network become master with
the formula (total num of nodes / 2 + 1) one side will always become
inaccessible?

On Mon, Jan 13, 2014 at 3:50 PM, Mark Walkom markw@campaignmonitor.comwrote:

The logs shows that a node was not contactable and so was removed from
the cluster.
That could be a load issue on the node, network issues, or something
else. Check the logs on the node that disconnected and see what you find.

If you leave the defaults, you risk the chance of a split-brain
situation, where nodes lose contact with each other and then decide to
become a master. This can cause you to end up with multiple masters in
multiple clusters, which is not good for data integrity.

Setting the min masters means that if a node cannot contact N masters, it
will refuse connections.

Regards,
Mark Walkom

Infrastructure Engineer
Campaign Monitor
email: markw@campaignmonitor.com
web: www.campaignmonitor.com

On 14 January 2014 10:46, Mohit Anchlia mohitanchlia@gmail.com wrote:

I am trying to understand why I should change it? This setting looks
like the right setting in most cases. Did you see anything out of ordinary
in that log message I pasted?

On Mon, Jan 13, 2014 at 3:40 PM, David Pilato david@pilato.fr wrote:

No it's not the default setting.
min master node = total num of nodes / 2 + 1

You have 5 nodes, 5 / 2 + 1 = 3

--
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet https://twitter.com/dadoonet | @elasticsearchfrhttps://twitter.com/elasticsearchfr

Le 14 janvier 2014 at 00:06:34, Mohit Anchlia (mohitanchlia@gmail.com//mohitanchlia@gmail.com)
a écrit:

Is the default setting 3? How do I go about selecting right
configuration setting for this property?

On Mon, Jan 13, 2014 at 3:01 PM, David Pilato david@pilato.fr wrote:

So I think you should first set minimum master node to 3. See
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/cluster-update-settings.html#_discovery

Probably you need to close your isolated node and restart it.

 --

David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet https://twitter.com/dadoonet | @elasticsearchfrhttps://twitter.com/elasticsearchfr

Le 13 janvier 2014 at 23:58:33, Mohit Anchlia (mohitanchlia@gmail.com//mohitanchlia@gmail.com)
a écrit:

I have 5 nodes. I am using all the default settings.

On Mon, Jan 13, 2014 at 2:32 PM, David Pilato david@pilato.fr wrote:

It sounds like a split brain issue or something.

How many nodes do you have?
Did you set minimum master node?

 --

David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet https://twitter.com/dadoonet | @elasticsearchfrhttps://twitter.com/elasticsearchfr

Le 13 janvier 2014 at 22:48:32, Mohit Anchlia (mohitanchlia@gmail.com//mohitanchlia@gmail.com)
a écrit:

Thanks! It looks like one of the nodes even though it was up still
it went out of cluster. Is there no retry?

[2014-01-02 17:55:01,148][INFO ][cluster.service ] [10.80.140.14]
master {new [10.80.140.14][267foffDRhO6zzXlHg3YKA][inet[/10.80.140.14:9300]]{group=group1},
previous [10.80.140.59][YDHawZWnRUKukXDIIT5MJQ][inet[/10.80.140.59:9300]]{group=group2}},
removed {[10.80.140.59][YDHawZWnRUKukXDIIT5MJQ][inet[/10.80.140.59:9300]]{group=group2},},
reason: zen-disco-master_failed
([10.80.140.59][YDHawZWnRUKukXDIIT5MJQ][inet[/10.80.140.59:9300
]]{group=group2})

[2014-01-02 17:55:02,158][INFO ][cluster.service ] [10.80.140.14]
removed {[10.80.140.13][6cf39fcASKSn6dNNsSXguA][inet[/10.80.140.13:9300]]{group=group2},},
reason:
zen-disco-node_failed([10.80.140.13][6cf39fcASKSn6dNNsSXguA][inet[/10.80.140.13:9300]]{group=group2}),
reason failed to ping, tried [3] times, each with maximum [30s] timeout

On Mon, Jan 13, 2014 at 1:34 PM, David Pilato david@pilato.frwrote:

Yellow means that some replicas are not allocated.
If you have only one node and have index set with one replica
(default), yellow is the expected state.

 --

David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet https://twitter.com/dadoonet | @elasticsearchfrhttps://twitter.com/elasticsearchfr

Le 13 janvier 2014 at 22:30:23, Mohit Anchlia (
mohitanchlia@gmail.com //mohitanchlia@gmail.com) a écrit:

Cluster state is showing yellow. What's the best way to find why
it's in yellow state?

It has been up and stable so far except when I looked today it shows
state turned into yellow.

You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAOT3TWpSSEVTpYA-DB8Hgfyo9WdR4bVRoKtQ9c_PC%2BFZ1aQiKg%40mail.gmail.com
.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/etPan.52d45be9.7c3dbd3d.376c%40MacBook-Air-de-David.local.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAOT3TWpLn_AgMKcTwQuGD-CHHe6ptg%3DRxaNGLPxFn3kroed%3DVQ%40mail.gmail.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/etPan.52d46968.5577f8e1.376c%40MacBook-Air-de-David.local.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAOT3TWrsU-k9u6hir2CWoobhnaOTD_3Bjk3Xf_ZjMDSBkB0DeQ%40mail.gmail.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/etPan.52d47036.77465f01.376c%40MacBook-Air-de-David.local.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAOT3TWqbrUh5p2t74Bc_6kL%3DxjAcLGrJzJ6wt_fRQqy6Dj-DKg%40mail.gmail.com
.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/etPan.52d47982.5e884adc.376c%40MacBook-Air-de-David.local
.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAOT3TWqx_KLF_p8nJ4m3On-nSvRncY9OdHa8KS1HDjcRHhfQhQ%40mail.gmail.com
.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAEM624aGEw8HqnMP--Fb-U%2BCjB58VBFb1bQenk_RmenNKAmMHg%40mail.gmail.com
.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAOT3TWrGJOS4%2BA6bm%3DKLm7aTDoQxX8TZGM7CgxVoGAeO%3D-XNLA%40mail.gmail.com
.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAEM624Z-OGv6nJpwB6OyNjX1wf%3Dk1n9J7BrB_M33X8dYkLVSYw%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Mohit Anchlia) #14

I didn't quite understand. What I am trying to find is that when default
formula of N / 2 + 1 I will always have more master nodes on one side
correct? And is my understanding incorrect that the other side that has
less master nodes than N /2 + 1 would simply not accept any requests? The
only point of increasing this value would be for performance reasons?

On Mon, Jan 13, 2014 at 3:58 PM, Mark Walkom markw@campaignmonitor.comwrote:

That depends on however many nodes lose contact, but it's a possibility
that if you lose 2 of your nodes at once they will just stop responding and
the other 3 will be ok.
However you maintain cluster integrity.

Regards,
Mark Walkom

Infrastructure Engineer
Campaign Monitor
email: markw@campaignmonitor.com
web: www.campaignmonitor.com

On 14 January 2014 10:55, Mohit Anchlia mohitanchlia@gmail.com wrote:

How would the nodes on the other side of the network become master with
the formula (total num of nodes / 2 + 1) one side will always become
inaccessible?

On Mon, Jan 13, 2014 at 3:50 PM, Mark Walkom markw@campaignmonitor.comwrote:

The logs shows that a node was not contactable and so was removed from
the cluster.
That could be a load issue on the node, network issues, or something
else. Check the logs on the node that disconnected and see what you find.

If you leave the defaults, you risk the chance of a split-brain
situation, where nodes lose contact with each other and then decide to
become a master. This can cause you to end up with multiple masters in
multiple clusters, which is not good for data integrity.

Setting the min masters means that if a node cannot contact N masters,
it will refuse connections.

Regards,
Mark Walkom

Infrastructure Engineer
Campaign Monitor
email: markw@campaignmonitor.com
web: www.campaignmonitor.com

On 14 January 2014 10:46, Mohit Anchlia mohitanchlia@gmail.com wrote:

I am trying to understand why I should change it? This setting looks
like the right setting in most cases. Did you see anything out of ordinary
in that log message I pasted?

On Mon, Jan 13, 2014 at 3:40 PM, David Pilato david@pilato.fr wrote:

No it's not the default setting.
min master node = total num of nodes / 2 + 1

You have 5 nodes, 5 / 2 + 1 = 3

--
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet https://twitter.com/dadoonet | @elasticsearchfrhttps://twitter.com/elasticsearchfr

Le 14 janvier 2014 at 00:06:34, Mohit Anchlia (mohitanchlia@gmail.com//mohitanchlia@gmail.com)
a écrit:

Is the default setting 3? How do I go about selecting right
configuration setting for this property?

On Mon, Jan 13, 2014 at 3:01 PM, David Pilato david@pilato.fr wrote:

So I think you should first set minimum master node to 3. See
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/cluster-update-settings.html#_discovery

Probably you need to close your isolated node and restart it.

 --

David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet https://twitter.com/dadoonet | @elasticsearchfrhttps://twitter.com/elasticsearchfr

Le 13 janvier 2014 at 23:58:33, Mohit Anchlia (mohitanchlia@gmail.com//mohitanchlia@gmail.com)
a écrit:

I have 5 nodes. I am using all the default settings.

On Mon, Jan 13, 2014 at 2:32 PM, David Pilato david@pilato.frwrote:

It sounds like a split brain issue or something.

How many nodes do you have?
Did you set minimum master node?

 --

David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet https://twitter.com/dadoonet | @elasticsearchfrhttps://twitter.com/elasticsearchfr

Le 13 janvier 2014 at 22:48:32, Mohit Anchlia (
mohitanchlia@gmail.com //mohitanchlia@gmail.com) a écrit:

Thanks! It looks like one of the nodes even though it was up
still it went out of cluster. Is there no retry?

[2014-01-02 17:55:01,148][INFO ][cluster.service ] [10.80.140.14]
master {new [10.80.140.14][267foffDRhO6zzXlHg3YKA][inet[/10.80.140.14:9300]]{group=group1},
previous [10.80.140.59][YDHawZWnRUKukXDIIT5MJQ][inet[/10.80.140.59:9300]]{group=group2}},
removed {[10.80.140.59][YDHawZWnRUKukXDIIT5MJQ][inet[/10.80.140.59:9300]]{group=group2},},
reason: zen-disco-master_failed
([10.80.140.59][YDHawZWnRUKukXDIIT5MJQ][inet[/10.80.140.59:9300
]]{group=group2})

[2014-01-02 17:55:02,158][INFO ][cluster.service ] [10.80.140.14]
removed {[10.80.140.13][6cf39fcASKSn6dNNsSXguA][inet[/10.80.140.13:9300]]{group=group2},},
reason:
zen-disco-node_failed([10.80.140.13][6cf39fcASKSn6dNNsSXguA][inet[/10.80.140.13:9300]]{group=group2}),
reason failed to ping, tried [3] times, each with maximum [30s] timeout

On Mon, Jan 13, 2014 at 1:34 PM, David Pilato david@pilato.frwrote:

Yellow means that some replicas are not allocated.
If you have only one node and have index set with one replica
(default), yellow is the expected state.

 --

David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet https://twitter.com/dadoonet | @elasticsearchfrhttps://twitter.com/elasticsearchfr

Le 13 janvier 2014 at 22:30:23, Mohit Anchlia (
mohitanchlia@gmail.com //mohitanchlia@gmail.com) a écrit:

Cluster state is showing yellow. What's the best way to find why
it's in yellow state?

It has been up and stable so far except when I looked today it
shows state turned into yellow.

You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAOT3TWpSSEVTpYA-DB8Hgfyo9WdR4bVRoKtQ9c_PC%2BFZ1aQiKg%40mail.gmail.com
.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/etPan.52d45be9.7c3dbd3d.376c%40MacBook-Air-de-David.local.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAOT3TWpLn_AgMKcTwQuGD-CHHe6ptg%3DRxaNGLPxFn3kroed%3DVQ%40mail.gmail.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/etPan.52d46968.5577f8e1.376c%40MacBook-Air-de-David.local.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAOT3TWrsU-k9u6hir2CWoobhnaOTD_3Bjk3Xf_ZjMDSBkB0DeQ%40mail.gmail.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/etPan.52d47036.77465f01.376c%40MacBook-Air-de-David.local.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAOT3TWqbrUh5p2t74Bc_6kL%3DxjAcLGrJzJ6wt_fRQqy6Dj-DKg%40mail.gmail.com
.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/etPan.52d47982.5e884adc.376c%40MacBook-Air-de-David.local
.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAOT3TWqx_KLF_p8nJ4m3On-nSvRncY9OdHa8KS1HDjcRHhfQhQ%40mail.gmail.com
.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAEM624aGEw8HqnMP--Fb-U%2BCjB58VBFb1bQenk_RmenNKAmMHg%40mail.gmail.com
.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAOT3TWrGJOS4%2BA6bm%3DKLm7aTDoQxX8TZGM7CgxVoGAeO%3D-XNLA%40mail.gmail.com
.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAEM624Z-OGv6nJpwB6OyNjX1wf%3Dk1n9J7BrB_M33X8dYkLVSYw%40mail.gmail.com
.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAOT3TWqhKdSvYLs%3Dy6eVzLjEfbYZa8z8Nj_ftEGt6u0yQmQkFg%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.


(David Pilato) #15

Default value for minimum master node is 1.
No formula here.

It was just an advice. Sorry if I was unclear. :slight_smile:

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 14 janv. 2014 à 20:58, Mohit Anchlia mohitanchlia@gmail.com a écrit :

I didn't quite understand. What I am trying to find is that when default formula of N / 2 + 1 I will always have more master nodes on one side correct? And is my understanding incorrect that the other side that has less master nodes than N /2 + 1 would simply not accept any requests? The only point of increasing this value would be for performance reasons?

On Mon, Jan 13, 2014 at 3:58 PM, Mark Walkom markw@campaignmonitor.com wrote:

That depends on however many nodes lose contact, but it's a possibility that if you lose 2 of your nodes at once they will just stop responding and the other 3 will be ok.
However you maintain cluster integrity.

Regards,
Mark Walkom

Infrastructure Engineer
Campaign Monitor
email: markw@campaignmonitor.com
web: www.campaignmonitor.com

On 14 January 2014 10:55, Mohit Anchlia mohitanchlia@gmail.com wrote:

How would the nodes on the other side of the network become master with the formula (total num of nodes / 2 + 1) one side will always become inaccessible?

On Mon, Jan 13, 2014 at 3:50 PM, Mark Walkom markw@campaignmonitor.com wrote:

The logs shows that a node was not contactable and so was removed from the cluster.
That could be a load issue on the node, network issues, or something else. Check the logs on the node that disconnected and see what you find.

If you leave the defaults, you risk the chance of a split-brain situation, where nodes lose contact with each other and then decide to become a master. This can cause you to end up with multiple masters in multiple clusters, which is not good for data integrity.

Setting the min masters means that if a node cannot contact N masters, it will refuse connections.

Regards,
Mark Walkom

Infrastructure Engineer
Campaign Monitor
email: markw@campaignmonitor.com
web: www.campaignmonitor.com

On 14 January 2014 10:46, Mohit Anchlia mohitanchlia@gmail.com wrote:

I am trying to understand why I should change it? This setting looks like the right setting in most cases. Did you see anything out of ordinary in that log message I pasted?

On Mon, Jan 13, 2014 at 3:40 PM, David Pilato david@pilato.fr wrote:

No it's not the default setting.
min master node = total num of nodes / 2 + 1

You have 5 nodes, 5 / 2 + 1 = 3

--
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet | @elasticsearchfr

Le 14 janvier 2014 at 00:06:34, Mohit Anchlia (mohitanchlia@gmail.com) a écrit:

Is the default setting 3? How do I go about selecting right configuration setting for this property?

On Mon, Jan 13, 2014 at 3:01 PM, David Pilato david@pilato.fr wrote:

So I think you should first set minimum master node to 3. See http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/cluster-update-settings.html#_discovery

Probably you need to close your isolated node and restart it.

--
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet | @elasticsearchfr

Le 13 janvier 2014 at 23:58:33, Mohit Anchlia (mohitanchlia@gmail.com) a écrit:

I have 5 nodes. I am using all the default settings.

On Mon, Jan 13, 2014 at 2:32 PM, David Pilato david@pilato.fr wrote:

It sounds like a split brain issue or something.

How many nodes do you have?
Did you set minimum master node?

--
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet | @elasticsearchfr

Le 13 janvier 2014 at 22:48:32, Mohit Anchlia (mohitanchlia@gmail.com) a écrit:

Thanks! It looks like one of the nodes even though it was up still it went out of cluster. Is there no retry?

[2014-01-02 17:55:01,148][INFO ][cluster.service ] [10.80.140.14] master {new [10.80.140.14][267foffDRhO6zzXlHg3YKA][inet[/10.80.140.14:9300]]{group=group1}, previous [10.80.140.59][YDHawZWnRUKukXDIIT5MJQ][inet[/10.80.140.59:9300]]{group=group2}}, removed {[10.80.140.59][YDHawZWnRUKukXDIIT5MJQ][inet[/10.80.140.59:9300]]{group=group2},}, reason: zen-disco-master_failed ([10.80.140.59][YDHawZWnRUKukXDIIT5MJQ][inet[/10.80.140.59:9300]]{group=group2})

[2014-01-02 17:55:02,158][INFO ][cluster.service ] [10.80.140.14] removed {[10.80.140.13][6cf39fcASKSn6dNNsSXguA][inet[/10.80.140.13:9300]]{group=group2},}, reason: zen-disco-node_failed([10.80.140.13][6cf39fcASKSn6dNNsSXguA][inet[/10.80.140.13:9300]]{group=group2}), reason failed to ping, tried [3] times, each with maximum [30s] timeout

On Mon, Jan 13, 2014 at 1:34 PM, David Pilato david@pilato.fr wrote:

Yellow means that some replicas are not allocated.
If you have only one node and have index set with one replica (default), yellow is the expected state.

--
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet | @elasticsearchfr

Le 13 janvier 2014 at 22:30:23, Mohit Anchlia (mohitanchlia@gmail.com) a écrit:

Cluster state is showing yellow. What's the best way to find why it's in yellow state?

It has been up and stable so far except when I looked today it shows state turned into yellow.

You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAOT3TWpSSEVTpYA-DB8Hgfyo9WdR4bVRoKtQ9c_PC%2BFZ1aQiKg%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/etPan.52d45be9.7c3dbd3d.376c%40MacBook-Air-de-David.local.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAOT3TWpLn_AgMKcTwQuGD-CHHe6ptg%3DRxaNGLPxFn3kroed%3DVQ%40mail.gmail.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/etPan.52d46968.5577f8e1.376c%40MacBook-Air-de-David.local.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAOT3TWrsU-k9u6hir2CWoobhnaOTD_3Bjk3Xf_ZjMDSBkB0DeQ%40mail.gmail.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/etPan.52d47036.77465f01.376c%40MacBook-Air-de-David.local.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAOT3TWqbrUh5p2t74Bc_6kL%3DxjAcLGrJzJ6wt_fRQqy6Dj-DKg%40mail.gmail.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/etPan.52d47982.5e884adc.376c%40MacBook-Air-de-David.local.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAOT3TWqx_KLF_p8nJ4m3On-nSvRncY9OdHa8KS1HDjcRHhfQhQ%40mail.gmail.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAEM624aGEw8HqnMP--Fb-U%2BCjB58VBFb1bQenk_RmenNKAmMHg%40mail.gmail.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAOT3TWrGJOS4%2BA6bm%3DKLm7aTDoQxX8TZGM7CgxVoGAeO%3D-XNLA%40mail.gmail.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAEM624Z-OGv6nJpwB6OyNjX1wf%3Dk1n9J7BrB_M33X8dYkLVSYw%40mail.gmail.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAOT3TWqhKdSvYLs%3Dy6eVzLjEfbYZa8z8Nj_ftEGt6u0yQmQkFg%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/87F44BA1-8F99-45C4-BF26-9DE7A209A6F1%40pilato.fr.
For more options, visit https://groups.google.com/groups/opt_out.


(Mohit Anchlia) #16

Ahh ..why wouldn't we have this set as default? N + 1 / 2 is fairly
commonly used in distributed dbs.

On Tue, Jan 14, 2014 at 12:00 PM, David Pilato david@pilato.fr wrote:

Default value for minimum master node is 1.
No formula here.

It was just an advice. Sorry if I was unclear. :slight_smile:

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 14 janv. 2014 à 20:58, Mohit Anchlia mohitanchlia@gmail.com a écrit :

I didn't quite understand. What I am trying to find is that when default
formula of N / 2 + 1 I will always have more master nodes on one side
correct? And is my understanding incorrect that the other side that has
less master nodes than N /2 + 1 would simply not accept any requests? The
only point of increasing this value would be for performance reasons?

On Mon, Jan 13, 2014 at 3:58 PM, Mark Walkom markw@campaignmonitor.comwrote:

That depends on however many nodes lose contact, but it's a possibility
that if you lose 2 of your nodes at once they will just stop responding and
the other 3 will be ok.
However you maintain cluster integrity.

Regards,
Mark Walkom

Infrastructure Engineer
Campaign Monitor
email: markw@campaignmonitor.com
web: www.campaignmonitor.com

On 14 January 2014 10:55, Mohit Anchlia mohitanchlia@gmail.com wrote:

How would the nodes on the other side of the network become master with
the formula (total num of nodes / 2 + 1) one side will always become
inaccessible?

On Mon, Jan 13, 2014 at 3:50 PM, Mark Walkom markw@campaignmonitor.comwrote:

The logs shows that a node was not contactable and so was removed from
the cluster.
That could be a load issue on the node, network issues, or something
else. Check the logs on the node that disconnected and see what you find.

If you leave the defaults, you risk the chance of a split-brain
situation, where nodes lose contact with each other and then decide to
become a master. This can cause you to end up with multiple masters in
multiple clusters, which is not good for data integrity.

Setting the min masters means that if a node cannot contact N masters,
it will refuse connections.

Regards,
Mark Walkom

Infrastructure Engineer
Campaign Monitor
email: markw@campaignmonitor.com
web: www.campaignmonitor.com

On 14 January 2014 10:46, Mohit Anchlia mohitanchlia@gmail.com wrote:

I am trying to understand why I should change it? This setting looks
like the right setting in most cases. Did you see anything out of ordinary
in that log message I pasted?

On Mon, Jan 13, 2014 at 3:40 PM, David Pilato david@pilato.fr wrote:

No it's not the default setting.
min master node = total num of nodes / 2 + 1

You have 5 nodes, 5 / 2 + 1 = 3

--
David Pilato | Technical Advocate | Elasticsearch.com
http://Elasticsearch.com

@dadoonet https://twitter.com/dadoonet | @elasticsearchfrhttps://twitter.com/elasticsearchfr

Le 14 janvier 2014 at 00:06:34, Mohit Anchlia (mohitanchlia@gmail.com//mohitanchlia@gmail.com)
a écrit:

Is the default setting 3? How do I go about selecting right
configuration setting for this property?

On Mon, Jan 13, 2014 at 3:01 PM, David Pilato david@pilato.frwrote:

So I think you should first set minimum master node to 3. See
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/cluster-update-settings.html#_discovery

Probably you need to close your isolated node and restart it.

 --

David Pilato | Technical Advocate | Elasticsearch.com
http://Elasticsearch.com

@dadoonet https://twitter.com/dadoonet | @elasticsearchfrhttps://twitter.com/elasticsearchfr

Le 13 janvier 2014 at 23:58:33, Mohit Anchlia (
mohitanchlia@gmail.com //mohitanchlia@gmail.com) a écrit:

I have 5 nodes. I am using all the default settings.

On Mon, Jan 13, 2014 at 2:32 PM, David Pilato david@pilato.frwrote:

It sounds like a split brain issue or something.

How many nodes do you have?
Did you set minimum master node?

 --

David Pilato | Technical Advocate | Elasticsearch.com
http://Elasticsearch.com

@dadoonet https://twitter.com/dadoonet | @elasticsearchfrhttps://twitter.com/elasticsearchfr

Le 13 janvier 2014 at 22:48:32, Mohit Anchlia (
mohitanchlia@gmail.com //mohitanchlia@gmail.com) a écrit:

Thanks! It looks like one of the nodes even though it was up
still it went out of cluster. Is there no retry?

[2014-01-02 17:55:01,148][INFO ][cluster.service ] [10.80.140.14]
master {new [10.80.140.14][267foffDRhO6zzXlHg3YKA][inet[/10.80.140.14:9300]]{group=group1},
previous [10.80.140.59][YDHawZWnRUKukXDIIT5MJQ][inet[/10.80.140.59:9300]]{group=group2}},
removed {[10.80.140.59][YDHawZWnRUKukXDIIT5MJQ][inet[/10.80.140.59:9300]]{group=group2},},
reason: zen-disco-master_failed
([10.80.140.59][YDHawZWnRUKukXDIIT5MJQ][inet[/10.80.140.59:9300
]]{group=group2})

[2014-01-02 17:55:02,158][INFO ][cluster.service ] [10.80.140.14]
removed {[10.80.140.13][6cf39fcASKSn6dNNsSXguA][inet[/10.80.140.13:9300]]{group=group2},},
reason:
zen-disco-node_failed([10.80.140.13][6cf39fcASKSn6dNNsSXguA][inet[/10.80.140.13:9300]]{group=group2}),
reason failed to ping, tried [3] times, each with maximum [30s] timeout

On Mon, Jan 13, 2014 at 1:34 PM, David Pilato david@pilato.frwrote:

Yellow means that some replicas are not allocated.
If you have only one node and have index set with one replica
(default), yellow is the expected state.

 --

David Pilato | Technical Advocate | Elasticsearch.com
http://Elasticsearch.com

@dadoonet https://twitter.com/dadoonet | @elasticsearchfrhttps://twitter.com/elasticsearchfr

Le 13 janvier 2014 at 22:30:23, Mohit Anchlia (
mohitanchlia@gmail.com //mohitanchlia@gmail.com) a écrit:

Cluster state is showing yellow. What's the best way to find
why it's in yellow state?

It has been up and stable so far except when I looked today it
shows state turned into yellow.

You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAOT3TWpSSEVTpYA-DB8Hgfyo9WdR4bVRoKtQ9c_PC%2BFZ1aQiKg%40mail.gmail.com
.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/etPan.52d45be9.7c3dbd3d.376c%40MacBook-Air-de-David.local.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAOT3TWpLn_AgMKcTwQuGD-CHHe6ptg%3DRxaNGLPxFn3kroed%3DVQ%40mail.gmail.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/etPan.52d46968.5577f8e1.376c%40MacBook-Air-de-David.local.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAOT3TWrsU-k9u6hir2CWoobhnaOTD_3Bjk3Xf_ZjMDSBkB0DeQ%40mail.gmail.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/etPan.52d47036.77465f01.376c%40MacBook-Air-de-David.local.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAOT3TWqbrUh5p2t74Bc_6kL%3DxjAcLGrJzJ6wt_fRQqy6Dj-DKg%40mail.gmail.com
.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/etPan.52d47982.5e884adc.376c%40MacBook-Air-de-David.local
.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAOT3TWqx_KLF_p8nJ4m3On-nSvRncY9OdHa8KS1HDjcRHhfQhQ%40mail.gmail.com
.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAEM624aGEw8HqnMP--Fb-U%2BCjB58VBFb1bQenk_RmenNKAmMHg%40mail.gmail.com
.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAOT3TWrGJOS4%2BA6bm%3DKLm7aTDoQxX8TZGM7CgxVoGAeO%3D-XNLA%40mail.gmail.com
.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAEM624Z-OGv6nJpwB6OyNjX1wf%3Dk1n9J7BrB_M33X8dYkLVSYw%40mail.gmail.com
.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAOT3TWqhKdSvYLs%3Dy6eVzLjEfbYZa8z8Nj_ftEGt6u0yQmQkFg%40mail.gmail.com
.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/87F44BA1-8F99-45C4-BF26-9DE7A209A6F1%40pilato.fr
.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAOT3TWpU_iDioGdxDCCJRhVTpc-nNfBYHdGLKUJoBATsRuaPiw%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Jörg Prante) #17

To be fair, the default minimum_master_node value is -1 (disabled).

Problem is, you do not know N when an ES node starts up and uses discovery

  • it can grow by time :wink:

Jörg

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoED55JCMB65gb-13pNxinqR0F1hDuVvh_dtjLNuyWB1iA%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Mohit Anchlia) #18

What's the problem we face when ES node starts up? Can you please elaborate?

On Tue, Jan 14, 2014 at 12:54 PM, joergprante@gmail.com <
joergprante@gmail.com> wrote:

To be fair, the default minimum_master_node value is -1 (disabled).

Problem is, you do not know N when an ES node starts up and uses discovery

  • it can grow by time :wink:

Jörg

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoED55JCMB65gb-13pNxinqR0F1hDuVvh_dtjLNuyWB1iA%40mail.gmail.com
.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAOT3TWoxQkYnB0cn-rvJG6d6f3q56L8W%3D%2B56akhWZHcNydkw%2Bw%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Jörg Prante) #19

The formula N+1/2 can not be used by ES as default for minimum_master_nodes
because for the variable N the value is unknown.

Jörg

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHzw1VxETHmmWqFK2wC7Fk0VVxywH6QR3qiNYB_7eDGww%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Mohit Anchlia) #20

Wouldn't the value increase as you add more nodes?

On Tue, Jan 14, 2014 at 1:22 PM, joergprante@gmail.com <
joergprante@gmail.com> wrote:

The formula N+1/2 can not be used by ES as default for
minimum_master_nodes because for the variable N the value is unknown.

Jörg

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHzw1VxETHmmWqFK2wC7Fk0VVxywH6QR3qiNYB_7eDGww%40mail.gmail.com
.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAOT3TWo5p0oXce7CAmr8OPRit_6LwVWuHyCSX%2BX%3Ds4Qve9sMHA%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.