Elastic, unable to set a cluster with three nodes

Elastic Stack Elasticsearch
1

Hello,
I triing to install elastic as cluster on three environments, where previuosly was installed and working.

i have a elasticsearch 7.8.1 zip package and this i have put and extracted on three machines (im in Windows ) and i set the same elasticsearch.yml file for all three nodes as:

cluster.name: elk-elastic-cluster
node.name: elk-elastic-cluster-node-1
path.data: G:\elasticsearch\data
path.logs: G:\elasticsearch\logs
network.host: 0.0.0.0
http.port: 9200
discovery.seed_hosts: ["pod-elk-006", "pod-elk-007", "pod-elk-008"]
cluster.initial_master_nodes: ["elk-elastic-cluster-node-1", "elk-elastic-cluster-node-2", "elk-elastic-cluster-node-3"]
logger.org.elasticsearch.discovery: TRACE
logger.org.elasticsearch.cluster.coordination: TRACE

when i start elastic on every machines i have in the log of first node:

[2021-01-21T17:01:11,209][WARN ][o.e.c.c.ClusterFormationFailureHelper] [elk-elastic-cluster-node-1] master not discovered or elected yet, an election requires at least 2 nodes with ids from [UlnGHt3XRsSZxZH0YEwVJw, m0QefzOjS7Kq4H0gbly-3A, iPRCXAlVTxCgQoNedae-7w], have discovered [{elk-elastic-cluster-node-1}{m0QefzOjS7Kq4H0gbly-3A}{VCyD3DGaTt2X2KALeTX95Q}{10.168.333.101}{10.168.333.101:9300}{dilmrt}{ml.machine_memory=8589328384, xpack.installed=true, transform.node=true, ml.max_open_jobs=20}] which is not a quorum; discovery will continue using [[::1]:9300, 10.168.333.102:9300, 10.168.333.103:9300] from hosts providers and [{elk-elastic-cluster-node-1}{m0QefzOjS7Kq4H0gbly-3A}{VCyD3DGaTt2X2KALeTX95Q}{10.168.333.101}{10.168.333.101:9300}{dilmrt}{ml.machine_memory=8589328384, xpack.installed=true, transform.node=true, ml.max_open_jobs=20}] from last-known cluster state; node term 5, last-accepted version 947 in term 5
[2021-01-21T17:01:11,859][TRACE][o.e.d.PeerFinder         ] [elk-elastic-cluster-node-1] probing master nodes from cluster state: nodes:
   {elk-elastic-cluster-node-1}{m0QefzOjS7Kq4H0gbly-3A}{VCyD3DGaTt2X2KALeTX95Q}{10.168.333.101}{10.168.333.101:9300}{dilmrt}{ml.machine_memory=8589328384, xpack.installed=true, transform.node=true, ml.max_open_jobs=20}, local
[2021-01-21T17:01:11,859][TRACE][o.e.d.PeerFinder         ] [elk-elastic-cluster-node-1] startProbe(10.168.333.101:9300) not probing local node
[2021-01-21T17:01:11,859][TRACE][o.e.d.SeedHostsResolver  ] [elk-elastic-cluster-node-1] resolved host [pod-elk-006] to [10.168.333.101:9300, [::1]:9300]
[2021-01-21T17:01:11,859][TRACE][o.e.d.SeedHostsResolver  ] [elk-elastic-cluster-node-1] resolved host [pod-elk-007] to [10.168.333.102:9300]
[2021-01-21T17:01:11,859][TRACE][o.e.d.SeedHostsResolver  ] [elk-elastic-cluster-node-1] resolved host [pod-elk-008] to [10.168.333.103:9300]
[2021-01-21T17:01:11,859][TRACE][o.e.d.PeerFinder         ] [elk-elastic-cluster-node-1] probing resolved transport addresses [[::1]:9300, 10.168.333.102:9300, 10.168.333.103:9300]
[2021-01-21T17:01:11,859][TRACE][o.e.d.PeerFinder         ] [elk-elastic-cluster-node-1] Peer{transportAddress=[::1]:9300, discoveryNode=null, peersRequestInFlight=false} attempting connection
[2021-01-21T17:01:11,859][TRACE][o.e.d.HandshakingTransportAddressConnector] [elk-elastic-cluster-node-1] [connectToRemoteMasterNode[[::1]:9300]] opening probe connection
[2021-01-21T17:01:11,869][TRACE][o.e.d.HandshakingTransportAddressConnector] [elk-elastic-cluster-node-1] [connectToRemoteMasterNode[[::1]:9300]] opened probe connection
[2021-01-21T17:01:11,870][TRACE][o.e.d.HandshakingTransportAddressConnector] [elk-elastic-cluster-node-1] [connectToRemoteMasterNode[[::1]:9300]] handshake successful: {elk-elastic-cluster-node-1}{m0QefzOjS7Kq4H0gbly-3A}{VCyD3DGaTt2X2KALeTX95Q}{10.168.333.101}{10.168.333.101:9300}{dilmrt}{ml.machine_memory=8589328384, ml.max_open_jobs=20, xpack.installed=true, transform.node=true}
[2021-01-21T17:01:11,871][DEBUG][o.e.d.PeerFinder         ] [elk-elastic-cluster-node-1] Peer{transportAddress=[::1]:9300, discoveryNode=null, peersRequestInFlight=false} connection failed
org.elasticsearch.transport.ConnectTransportException: [elk-elastic-cluster-node-1][10.168.333.101:9300] local node found
        at org.elasticsearch.discovery.HandshakingTransportAddressConnector$1$1$1.innerOnResponse(HandshakingTransportAddressConnector.java:105) [elasticsearch-7.8.1.jar:7.8.1]
        [...cutted here...]
[2021-01-21T17:01:12,865][TRACE][o.e.d.PeerFinder         ] [elk-elastic-cluster-node-1] probing master nodes from cluster state: nodes:
   {elk-elastic-cluster-node-1}{m0QefzOjS7Kq4H0gbly-3A}{VCyD3DGaTt2X2KALeTX95Q}{10.168.333.101}{10.168.333.101:9300}{dilmrt}{ml.machine_memory=8589328384, xpack.installed=true, transform.node=true, ml.max_open_jobs=20}, local

[2021-01-21T17:01:12,865][TRACE][o.e.d.PeerFinder         ] [elk-elastic-cluster-node-1] startProbe(10.168.333.101:9300) not probing local node
[2021-01-21T17:01:12,865][TRACE][o.e.d.SeedHostsResolver  ] [elk-elastic-cluster-node-1] resolved host [pod-elk-006] to [10.168.333.101:9300, [::1]:9300]
[2021-01-21T17:01:12,865][TRACE][o.e.d.SeedHostsResolver  ] [elk-elastic-cluster-node-1] resolved host [pod-elk-007] to [10.168.333.102:9300]
[2021-01-21T17:01:12,865][TRACE][o.e.d.SeedHostsResolver  ] [elk-elastic-cluster-node-1] resolved host [pod-elk-008] to [10.168.333.103:9300]
[2021-01-21T17:01:12,871][TRACE][o.e.d.PeerFinder         ] [elk-elastic-cluster-node-1] probing resolved transport addresses [[::1]:9300, 10.168.333.102:9300, 10.168.333.103:9300]
[2021-01-21T17:01:12,871][TRACE][o.e.d.PeerFinder         ] [elk-elastic-cluster-node-1] Peer{transportAddress=[::1]:9300, discoveryNode=null, peersRequestInFlight=false} attempting connection
[2021-01-21T17:01:12,871][TRACE][o.e.d.HandshakingTransportAddressConnector] [elk-elastic-cluster-node-1] [connectToRemoteMasterNode[[::1]:9300]] opening probe connection
[2021-01-21T17:01:12,875][TRACE][o.e.d.HandshakingTransportAddressConnector] [elk-elastic-cluster-node-1] [connectToRemoteMasterNode[[::1]:9300]] opened probe connection
[2021-01-21T17:01:12,876][TRACE][o.e.d.HandshakingTransportAddressConnector] [elk-elastic-cluster-node-1] [connectToRemoteMasterNode[[::1]:9300]] handshake successful: {elk-elastic-cluster-node-1}{m0QefzOjS7Kq4H0gbly-3A}{VCyD3DGaTt2X2KALeTX95Q}{10.168.333.101}{10.168.333.101:9300}{dilmrt}{ml.machine_memory=8589328384, ml.max_open_jobs=20, xpack.installed=true, transform.node=true}
[2021-01-21T17:01:12,881][DEBUG][o.e.d.PeerFinder         ] [elk-elastic-cluster-node-1] Peer{transportAddress=[::1]:9300, discoveryNode=null, peersRequestInFlight=false} connection failed
org.elasticsearch.transport.ConnectTransportException: [elk-elastic-cluster-node-1][10.168.333.101:9300] local node found
        at org.elasticsearch.discovery.HandshakingTransportAddressConnector$1$1$1.innerOnResponse(HandshakingTransportAddressConnector.java:105) [elasticsearch-7.8.1.jar:7.8.1]
        at org.elasticsearch.discovery.HandshakingTransportAddressConnector$1$1$1.innerOnResponse(HandshakingTransportAddressConnector.java:95) [elasticsearch-7.8.1.jar:7.8.1]
        [...cutted here...]]
[2021-01-21T17:01:13,867][DEBUG][o.e.d.PeerFinder         ] [elk-elastic-cluster-node-1] Peer{transportAddress=10.168.333.102:9300, discoveryNode=null, peersRequestInFlight=false} connection failed
org.elasticsearch.transport.ConnectTransportException: [][10.168.333.102:9300] connect_timeout[3s]
        at org.elasticsearch.transport.TcpTransport$ChannelsConnectedListener.onTimeout(TcpTransport.java:972) ~[elasticsearch-7.8.1.jar:7.8.1]
        [...cutted here...]]
2

log on second node:

[2021-01-21T17:08:38,525][WARN ][o.e.c.c.ClusterFormationFailureHelper] [elk-elastic-cluster-node-2] master not discovered yet, this node has not previously joined a bootstrapped (v7+) cluster, and this node must discover master-eligible nodes [elk-elastic-cluster-node-1, elk-elastic-cluster-node-2, elk-elastic-cluster-node-3] to bootstrap a cluster: have discovered [{elk-elastic-cluster-node-2}{nQ6ief-mSXa-wzly0xmCJw}{mqkTF7f2TRK_ybXHSHJJiw}{10.168.333.102}{10.168.333.102:9300}{dilmrt}{ml.machine_memory=8589328384, xpack.installed=true, transform.node=true, ml.max_open_jobs=20}]; discovery will continue using [10.168.333.101:9300, [::1]:9300, 10.168.333.103:9300] from hosts providers and [{elk-elastic-cluster-node-2}{nQ6ief-mSXa-wzly0xmCJw}{mqkTF7f2TRK_ybXHSHJJiw}{10.168.333.102}{10.168.333.102:9300}{dilmrt}{ml.machine_memory=8589328384, xpack.installed=true, transform.node=true, ml.max_open_jobs=20}] from last-known cluster state; node term 0, last-accepted version 0 in term 0

on the third:

[2021-01-21T17:01:11,209][WARN ][o.e.c.c.ClusterFormationFailureHelper] [elk-elastic-cluster-node-1] master not discovered or elected yet, an election requires at least 2 nodes with ids from [UlnGHt3XRsSZxZH0YEwVJw, m0QefzOjS7Kq4H0gbly-3A, iPRCXAlVTxCgQoNedae-7w], have discovered [{elk-elastic-cluster-node-1}{m0QefzOjS7Kq4H0gbly-3A}{VCyD3DGaTt2X2KALeTX95Q}{10.168.333.101}{10.168.333.101:9300}{dilmrt}{ml.machine_memory=8589328384, xpack.installed=true, transform.node=true, ml.max_open_jobs=20}] which is not a quorum; discovery will continue using [[::1]:9300, 10.168.333.102:9300, 10.168.333.103:9300] from hosts providers and [{elk-elastic-cluster-node-1}{m0QefzOjS7Kq4H0gbly-3A}{VCyD3DGaTt2X2KALeTX95Q}{10.168.333.101}{10.168.333.101:9300}{dilmrt}{ml.machine_memory=8589328384, xpack.installed=true, transform.node=true, ml.max_open_jobs=20}] from last-known cluster state; node term 5, last-accepted version 947 in term 5
[2021-01-21T17:01:11,859][TRACE][o.e.d.PeerFinder         ] [elk-elastic-cluster-node-1] probing master nodes from cluster state: nodes:
   {elk-elastic-cluster-node-1}{m0QefzOjS7Kq4H0gbly-3A}{VCyD3DGaTt2X2KALeTX95Q}{10.168.333.101}{10.168.333.101:9300}{dilmrt}{ml.machine_memory=8589328384, xpack.installed=true, transform.node=true, ml.max_open_jobs=20}, local

[2021-01-21T17:01:11,859][TRACE][o.e.d.PeerFinder         ] [elk-elastic-cluster-node-1] startProbe(10.168.333.101:9300) not probing local node
[2021-01-21T17:01:11,859][TRACE][o.e.d.SeedHostsResolver  ] [elk-elastic-cluster-node-1] resolved host [pod-elk-006] to [10.168.333.101:9300, [::1]:9300]
[2021-01-21T17:01:11,859][TRACE][o.e.d.SeedHostsResolver  ] [elk-elastic-cluster-node-1] resolved host [pod-elk-007] to [10.168.333.102:9300]
[2021-01-21T17:01:11,859][TRACE][o.e.d.SeedHostsResolver  ] [elk-elastic-cluster-node-1] resolved host [pod-elk-008] to [10.168.333.103:9300]
[2021-01-21T17:01:11,859][TRACE][o.e.d.PeerFinder         ] [elk-elastic-cluster-node-1] probing resolved transport addresses [[::1]:9300, 10.168.333.102:9300, 10.168.333.103:9300]
[2021-01-21T17:01:11,859][TRACE][o.e.d.PeerFinder         ] [elk-elastic-cluster-node-1] Peer{transportAddress=[::1]:9300, discoveryNode=null, peersRequestInFlight=false} attempting connection
[2021-01-21T17:01:11,859][TRACE][o.e.d.HandshakingTransportAddressConnector] [elk-elastic-cluster-node-1] [connectToRemoteMasterNode[[::1]:9300]] opening probe connection
[2021-01-21T17:01:11,869][TRACE][o.e.d.HandshakingTransportAddressConnector] [elk-elastic-cluster-node-1] [connectToRemoteMasterNode[[::1]:9300]] opened probe connection
[2021-01-21T17:01:11,870][TRACE][o.e.d.HandshakingTransportAddressConnector] [elk-elastic-cluster-node-1] [connectToRemoteMasterNode[[::1]:9300]] handshake successful: {elk-elastic-cluster-node-1}{m0QefzOjS7Kq4H0gbly-3A}{VCyD3DGaTt2X2KALeTX95Q}{10.168.333.101}{10.168.333.101:9300}{dilmrt}{ml.machine_memory=8589328384, ml.max_open_jobs=20, xpack.installed=true, transform.node=true}
[2021-01-21T17:01:11,871][DEBUG][o.e.d.PeerFinder         ] [elk-elastic-cluster-node-1] Peer{transportAddress=[::1]:9300, discoveryNode=null, peersRequestInFlight=false} connection failed
org.elasticsearch.transport.ConnectTransportException: [elk-elastic-cluster-node-1][10.168.333.101:9300] local node found
        at org.elasticsearch.discovery.HandshakingTransportAddressConnector$1$1$1.innerOnResponse(HandshakingTransportAddressConnector.java:105) [elasticsearch-7.8.1.jar:7.8.1]
        [...cutted here..]]
[2021-01-21T17:01:12,865][TRACE][o.e.d.PeerFinder         ] [elk-elastic-cluster-node-1] probing master nodes from cluster state: nodes:
   {elk-elastic-cluster-node-1}{m0QefzOjS7Kq4H0gbly-3A}{VCyD3DGaTt2X2KALeTX95Q}{10.168.333.101}{10.168.333.101:9300}{dilmrt}{ml.machine_memory=8589328384, xpack.installed=true, transform.node=true, ml.max_open_jobs=20}, local

[2021-01-21T17:01:12,865][TRACE][o.e.d.PeerFinder         ] [elk-elastic-cluster-node-1] startProbe(10.168.333.101:9300) not probing local node
[2021-01-21T17:01:12,865][TRACE][o.e.d.SeedHostsResolver  ] [elk-elastic-cluster-node-1] resolved host [pod-elk-006] to [10.168.333.101:9300, [::1]:9300]
[2021-01-21T17:01:12,865][TRACE][o.e.d.SeedHostsResolver  ] [elk-elastic-cluster-node-1] resolved host [pod-elk-007] to [10.168.333.102:9300]
[2021-01-21T17:01:12,865][TRACE][o.e.d.SeedHostsResolver  ] [elk-elastic-cluster-node-1] resolved host [pod-elk-008] to [10.168.333.103:9300]
[2021-01-21T17:01:12,871][TRACE][o.e.d.PeerFinder         ] [elk-elastic-cluster-node-1] probing resolved transport addresses [[::1]:9300, 10.168.333.102:9300, 10.168.333.103:9300]
[2021-01-21T17:01:12,871][TRACE][o.e.d.PeerFinder         ] [elk-elastic-cluster-node-1] Peer{transportAddress=[::1]:9300, discoveryNode=null, peersRequestInFlight=false} attempting connection
[2021-01-21T17:01:12,871][TRACE][o.e.d.HandshakingTransportAddressConnector] [elk-elastic-cluster-node-1] [connectToRemoteMasterNode[[::1]:9300]] opening probe connection
[2021-01-21T17:01:12,875][TRACE][o.e.d.HandshakingTransportAddressConnector] [elk-elastic-cluster-node-1] [connectToRemoteMasterNode[[::1]:9300]] opened probe connection
[2021-01-21T17:01:12,876][TRACE][o.e.d.HandshakingTransportAddressConnector] [elk-elastic-cluster-node-1] [connectToRemoteMasterNode[[::1]:9300]] handshake successful: {elk-elastic-cluster-node-1}{m0QefzOjS7Kq4H0gbly-3A}{VCyD3DGaTt2X2KALeTX95Q}{10.168.333.101}{10.168.333.101:9300}{dilmrt}{ml.machine_memory=8589328384, ml.max_open_jobs=20, xpack.installed=true, transform.node=true}
[2021-01-21T17:01:12,881][DEBUG][o.e.d.PeerFinder         ] [elk-elastic-cluster-node-1] Peer{transportAddress=[::1]:9300, discoveryNode=null, peersRequestInFlight=false} connection failed
org.elasticsearch.transport.ConnectTransportException: [elk-elastic-cluster-node-1][10.168.333.101:9300] local node found
        at org.elasticsearch.discovery.HandshakingTransportAddressConnector$1$1$1.innerOnResponse(HandshakingTransportAddressConnector.java:105) [elasticsearch-7.8.1.jar:7.8.1]
        [...cutted here..]
[2021-01-21T17:01:13,867][DEBUG][o.e.d.PeerFinder         ] [elk-elastic-cluster-node-1] Peer{transportAddress=10.168.333.102:9300, discoveryNode=null, peersRequestInFlight=false} connection failed
org.elasticsearch.transport.ConnectTransportException: [][10.168.333.102:9300] connect_timeout[3s]
        at org.elasticsearch.transport.TcpTransport$ChannelsConnectedListener.onTimeout(TcpTransport.java:972) ~[elasticsearch-7.8.1.jar:7.8.1]
 [...cutted here..]
3

Can anyone help me, I've done hundreds of tries and searches but I can't get it to work
Thanks

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.