No cluster health with 2 nodes


#1

Hi !

I created a script to automate the installation of elasticsearch 5.
It works fine.
But now I have a problem with elasticsearch: I can't have the cluster health and I have several errors.

first the elasticsearch config:

network.bind_host: 0.0.0.0
network.publish_host: 10.136.1.123 (the ip of the host)
discovery.zen.minimum_master_nodes: 2
node.name: Aquaman
node.master: true
node.data: true
discovery.zen.ping.unicast.hosts: ["liprd071.adellego.com", "liprd073.adellego.com", "liprd074.adellego.com"] (not the same for all the node)

The thing is when I put curl -XGET 'localhost:9200/_cluster/health?pretty', I've got nothing (It doesn't stop)

And this is a part of the log:
[2016-11-16T14:23:29,768][ERROR][o.e.g.GatewayService ] [Superman] unexpected failure during [local-gateway-elected-state]
org.elasticsearch.cluster.NotMasterException: no longer master. source: [local-gateway-elected-state]
[2016-11-16T14:23:29,768][INFO ][o.e.g.GatewayService ] [Superman] metadata state not restored, reason: failed to updated cluster state
[2016-11-16T14:23:32,763][INFO ][o.e.c.s.ClusterService ] [Superman] new_master {Superman}{EWHY0pBdRS6jEyTmE_HJdw}{xybavZhcR0yZ9wDpjHUVTQ}{10.136.1.115}{10.136.1.115:9300}, reason: zen-disco-elected-as-master ([1] nodes joined)[{l1BSU3u}{l1BSU3uLRwiA4Jelt-LKYw}{vC6v6nYcQTiwONQOSkoz8w}{127.0.0.1}{127.0.0.1:9300}]
[2016-11-16T14:24:02,763][WARN ][o.e.d.z.p.PublishClusterStateAction] [Superman] timed out waiting for all nodes to process published state [21] (timeout [30s], pending nodes: [{l1BSU3u}{l1BSU3uLRwiA4Jelt-LKYw}{vC6v6nYcQTiwONQOSkoz8w}{127.0.0.1}{127.0.0.1:9300}])
[2016-11-16T14:24:02,764][WARN ][o.e.c.s.ClusterService ] [Superman] cluster state update task [zen-disco-elected-as-master ([1] nodes joined)[{l1BSU3u}{l1BSU3uLRwiA4Jelt-LKYw}{vC6v6nYcQTiwONQOSkoz8w}{127.0.0.1}{127.0.0.1:9300}]] took [30s] above the warn threshold of 30s
[2016-11-16T14:24:02,764][WARN ][o.e.d.z.ZenDiscovery ] [Superman] discovered [{Superman}{EWHY0pBdRS6jEyTmE_HJdw}{xybavZhcR0yZ9wDpjHUVTQ}{10.136.1.115}{10.136.1.115:9300}] which is also master but with an older cluster_state, telling [{Superman}{EWHY0pBdRS6jEyTmE_HJdw}{xybavZhcR0yZ9wDpjHUVTQ}{10.136.1.115}{10.136.1.115:9300}] to rejoin the cluster ([via a new cluster state])
[2016-11-16T14:24:02,765][WARN ][o.e.d.z.ZenDiscovery ] [Superman] received a request to rejoin the cluster from [EWHY0pBdRS6jEyTmE_HJdw], current nodes: {{Superman}{EWHY0pBdRS6jEyTmE_HJdw}{xybavZhcR0yZ9wDpjHUVTQ}{10.136.1.115}{10.136.1.115:9300},{Aquaman}{fwYilkxgQgWSMsk673R6ww}{YVly9M3GTCaKNwTgugxV8A}{10.136.1.123}{10.136.1.123:9300},{l1BSU3u}{l1BSU3uLRwiA4Jelt-LKYw}{vC6v6nYcQTiwONQOSkoz8w}{127.0.0.1}{127.0.0.1:9300},}
[2016-11-16T14:24:02,773][ERROR][o.e.g.GatewayService ] [Superman] unexpected failure during [local-gateway-elected-state]
org.elasticsearch.cluster.NotMasterException: no longer master. source: [local-gateway-elected-state]
[2016-11-16T14:24:02,773][INFO ][o.e.g.GatewayService ] [Superman] metadata state not restored, reason: failed to updated cluster state
[2016-11-16T14:24:05,890][INFO ][o.e.c.s.ClusterService ] [Superman] new_master {Superman}{EWHY0pBdRS6jEyTmE_HJdw}{xybavZhcR0yZ9wDpjHUVTQ}{10.136.1.115}{10.136.1.115:9300}, reason: zen-disco-elected-as-master ([1] nodes joined)[{Aquaman}{fwYilkxgQgWSMsk673R6ww}{YVly9M3GTCaKNwTgugxV8A}{10.136.1.123}{10.136.1.123:9300}]

Superman is the first node and Aquaman the second.
What have I done wrong ?
Thank you !


(Xavier Facq) #2

You should read : https://www.elastic.co/guide/en/elasticsearch/reference/5.0/modules-discovery-zen.html#master-election

This setting must be set to a quorum of your master eligible nodes. It is recommended to avoid having 
only two         master eligible nodes, since a quorum of two is two. Therefore, a loss of either master 
eligible node will result in an inoperable cluster.

#3

You're not help I deserved but the I help I needed !

Thanks a lot, Cluster green !


(Xavier Facq) #4

:thumbsup:


(system) #5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.