ES 2.3.3 node cannot join the master node

Elastic Stack Elasticsearch
1

Hi,
I configured simple two-node cluster (for test purposes) with one master node. It's Red Hat servers. Unfortunately second node cannot join to the master node. Here is configuration of master node:

cluster.name: elksandpit
node.name: "sandpit1"
node.master: true
node.data: true
path.data: /var/opt/elk_storage/data
path.work: /var/opt/elk_storage/work
path.logs: /var/opt/elk_storage/logs
bootstrap.mlockall: true
discovery.zen.ping.multicast.enabled: false
discovery.zen.ping.unicast.hosts: ["127.0.0.1"]
transport.tcp.port: 9300
network.publish_host: aa.bbc.ccc.228  //I am hiding IP in post for privacy

And here is configuration of second node, this node is design only for Kibana:

cluster.name: elksandpit
node.name: "kibana"
node.master: false
node.data: false
bootstrap.mlockall: true
discovery.zen.ping.multicast.enabled: false
discovery.zen.ping.unicast.hosts: ["aa.bbb.ccc.228:9300"]
transport.tcp.connect_timeout: 2m
network.publish_host: aa.bbb.ccc.226

When I'm executing this from second node (not master):

curl -XGET localhost:9200/_cat/health
error appears:
{"error":{"root_cause":[{"type":"master_not_discovered_exception","reason":null}],"type":"master_not_discovered_exception","reason":null},"status":503}

MasterNotDiscoveredException[null]
        at org.elasticsearch.action.support.master.TransportMasterNodeAction$AsyncSingleAction$5.onTimeout(TransportMasterNodeAction.java:226)
        at org.elasticsearch.cluster.ClusterStateObserver$ObserverClusterStateListener.onTimeout(ClusterStateObserver.java:236)
        at org.elasticsearch.cluster.service.InternalClusterService$NotifyTimeout.run(InternalClusterService.java:804)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)

If somebody could give any hint. Thanks.

2

You do not needs this in 2.X, multicast doesn't exist in core.

And this is the default, so no need to specify it either.

Are both hosts in the same network? Can you curl aa.bbc.ccc.228:9200 from the kibana host?

3

network.host is not set here. So it listens only on 127.0.0.1

My 2 cents

4

Hi,
thank you for replaying
When I'm executing
curl aa.bbb.ccc.228:9200

from kibana host I'm getting
curl: (7) couldn't connect to host

Is setting network.host option (+removing unnecessary configuration entries) on both machines remedy for my issue ?

(Just asking, not checking this, because I lost root privileges for one day or two and waiting till our infrastucture grant me root again....)
Thanks!

5

It will help!