# Unable to communicate between 2 master nodes creating cluster issue **URL:** \ **Category:** Elasticsearch\ **Created:** [March 30, 2023, 11:36am UTC](https://discuss.elastic.co/t/unable-to-communicate-between-2-master-nodes-creating-cluster-issue/328931 "2023-03-30T11:36:35Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![devarajsit](https://avatars.discourse-cdn.com/v4/letter/d/e47c2d/32.png) [@devarajsit](https://discuss.elastic.co/u/devarajsit)\ **Post date:** [March 30, 2023, 11:36am UTC](https://discuss.elastic.co/t/unable-to-communicate-between-2-master-nodes-creating-cluster-issue/328931/1 "2023-03-30T11:36:35Z")
I tried creating a new Elasticsearch cluster with 3 master nodes, 3 data nodes and 2 clients. All the 3 master nodes status is in Running but when i verified the logs of the third node, it says time out connecting to 1st node: ```auto {"type": "server", "timestamp": "2023-03-30T11:34:24,460Z", "level": "WARN", "component": "o.e.c.c.ClusterFormationFailureHelper", "cluster.name": "elasticsearch", "node.name": "elasticsearch-master-2", "message": "master not discovered or elected yet, an election requires at least 2 nodes with ids from [gp82BsoETGyQxrVuq4KUDg, va3LZXpBTe6m2pdcE0nqvg, CsWX-u7XR8u2jNQ4AeD1Eg], have discovered possible quorum [{elasticsearch-master-2}{va3LZXpBTe6m2pdcE0nqvg}{LuzbJdgQTOOVnXQO7NRW_w}{10.237.14.89}{10.237.14.89:9300}{ilmr}, {elasticsearch-master-1}{gp82BsoETGyQxrVuq4KUDg}{o0I5HcIfTBmFDXUJiWqcng}{10.236.204.54}{10.236.204.54:9300}{ilmr}]; discovery will continue using [10.236.204.54:9300, 10.233.173.202:9300] from hosts providers and [{elasticsearch-master-2}{va3LZXpBTe6m2pdcE0nqvg}{LuzbJdgQTOOVnXQO7NRW_w}{10.237.14.89}{10.237.14.89:9300}{ilmr}] from last-known cluster state; node term 5, last-accepted version 59 in term 5" } {"type": "server", "timestamp": "2023-03-30T11:34:26,391Z", "level": "WARN", "component": "o.e.d.PeerFinder", "cluster.name": "elasticsearch", "node.name": "elasticsearch-master-2", "message": "address [10.233.173.202:9300], node [null], requesting [false] connection failed: [][10.233.173.202:9300] connect_timeout[30s]" } ``` When i executed the curl to cat nodes. i can see 2 nodes are there as part of the cluster: 10.225.3.47 3 75 0 0.00 0.02 0.03 cdfhilrstw - elasticsearch-data-1 10.233.173.204 10 75 0 0.18 0.09 0.02 cdfhilrstw - elasticsearch-data-0 10.233.173.202 38 83 1 0.18 0.09 0.02 ilmr \* elasticsearch-master-0 10.236.204.54 64 82 1 0.00 0.02 0.02 ilmr - elasticsearch-master-1 elasticsearch-master-2 and elasticsearch-data-2 are not joining to cluster. Any help would be appreciated. ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [March 30, 2023, 11:54am UTC](https://discuss.elastic.co/t/unable-to-communicate-between-2-master-nodes-creating-cluster-issue/328931/2 "2023-03-30T11:54:42Z")
It would help if you shared which version of Elasticsearch you are running and the configuration of the nodes. ---
**Author:** ![devarajsit](https://avatars.discourse-cdn.com/v4/letter/d/e47c2d/32.png) [@devarajsit](https://discuss.elastic.co/u/devarajsit)\ **Post date:** [March 30, 2023, 11:59am UTC](https://discuss.elastic.co/t/unable-to-communicate-between-2-master-nodes-creating-cluster-issue/328931/3 "2023-03-30T11:59:10Z")
Hi Christian, Thanks for replying. Version is "7.17.1" Let me know what configuration you need for the nodes? ```auto cluster.name: elasticsearch action.auto_create_index: true xpack.security.enabled: false xpack.security.transport.ssl.enabled: false node.data: ${NODE_DATA:true} node.master: ${NODE_MASTER:true} # vm.max_map_count: 262144 node.name: ${HOSTNAME} network.host: 0.0.0.0 http.host: 0.0.0.0 # see https://github.com/elastic/elasticsearch-definitive-guide/pull/679 processors: ${PROCESSORS:} # avoid split-brain w/ a minimum consensus of two masters plus a data node gateway.expected_master_nodes: ${EXPECTED_MASTER_NODES:1} gateway.expected_data_nodes: ${EXPECTED_DATA_NODES:1} gateway.recover_after_time: ${RECOVER_AFTER_TIME:5m} gateway.recover_after_master_nodes: ${RECOVER_AFTER_MASTER_NODES:1} gateway.recover_after_data_nodes: ${RECOVER_AFTER_DATA_NODES:1} ``` ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [March 30, 2023, 12:07pm UTC](https://discuss.elastic.co/t/unable-to-communicate-between-2-master-nodes-creating-cluster-issue/328931/4 "2023-03-30T12:07:53Z")
How are you [bootstrapping the cluster](https://www.elastic.co/guide/en/elasticsearch/reference/8.6/modules-discovery-bootstrap-cluster.html)? Are you not setting `discovery.seed_hosts` or `cluster.initial_master_nodes` for any of the nodes? I would also recommend running 3 master eligible nodes as [having just 2 does not give you any high availability](https://www.elastic.co/guide/en/elasticsearch/reference/8.6/high-availability-cluster-small-clusters.html). ---
**Author:** ![devarajsit](https://avatars.discourse-cdn.com/v4/letter/d/e47c2d/32.png) [@devarajsit](https://discuss.elastic.co/u/devarajsit)\ **Post date:** [March 30, 2023, 12:31pm UTC](https://discuss.elastic.co/t/unable-to-communicate-between-2-master-nodes-creating-cluster-issue/328931/5 "2023-03-30T12:31:54Z")
In the configuration we have discovery.seed\_hosts which is a kubernetes service and will take all the three nodes ```auto - name: discovery.seed_hosts value: elasticsearch-discovery ======================== elasticsearch-discovery 10.233.173.207:9300,10.236.204.55:9300,10.237.14.92:9300 3h22m ``` I have modified the master from 2 to 1 since i was getting issue with cluster formation. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [April 27, 2023, 12:32pm UTC](https://discuss.elastic.co/t/unable-to-communicate-between-2-master-nodes-creating-cluster-issue/328931/6 "2023-04-27T12:32:21Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.