Hello Team,
I am trying to form a cluster with 2 nodes in DEV environment, please find the below details and help me to resolve the issue.
node1 configuration
cluster.name: my-app
node.name: node1
network.host: 158.155.72.11
http.port: 9200
transport.host: 158.155.72.11
transport.port: 9300
discovery.seed_hosts: ["158.155.72.11:9300","158.155.201.245:9302"]
cluster.initial_master_nodes: ["node1","node2"]
node 2 configuration
cluster.name: my-app
node.name: node2
network.host: 158.155.201.245
http.port: 9202
transport.host: 158.155.201.245
transport.port: 9302
discovery.seed_hosts: ["158.155.201.245:9302","158.155.72.11:9300"]
cluster.initial_master_nodes: ["node1","node2"]
node1 log
[2020-06-26T16:42:10,685][WARN ][o.e.t.TcpTransport ] [node1] exception caught on transport layer [Netty4TcpChannel{localAddress=/158.155.72.11:9300, remoteAddress=/158.155.201.245:51258}], closing connection
io.netty.handler.codec.DecoderException: javax.net.ssl.SSLHandshakeException: No available authentication scheme
** at** io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:475) ~[netty-codec-4.1.38.Final.jar:4.1.38.Final]
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:283) ~[netty-codec-4.1.38.Final.jar:4.1.38.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374) [netty-transport-4.1.38.Final.jar:4.1.38.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360) [netty-transport-4.1.38.Final.jar:4.1.38.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352) [netty-transport-4.1.38.Final.jar:4.1.38.Final]
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1421) [netty-transport-4.1.38.Final.jar:4.1.38.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374) [netty-transport-4.1.38.Final.jar:4.1.38.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360) [netty-transport-4.1.38.Final.jar:4.1.38.Final]
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:930) [netty-transport-4.1.38.Final.jar:4.1.38.Final]
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) [netty-transport-4.1.38.Final.jar:4.1.38.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:697) [netty-transport-4.1.38.Final.jar:4.1.38.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:597) [netty-transport-4.1.38.Final.jar:4.1.38.Final]
at io.netty.channel.nio.NioEventLoop.processSele
node2 log
[2020-06-26T06:42:10,375][WARN ][o.e.c.c.ClusterFormationFailureHelper] [node2] master not discovered or elected yet, an election requires a node with id [PSVRfIbvR7mhd987JGVNEg], have discovered [{node2}{npszcdMaSQOnpLvz8r1gjw}{8ukSXP07STm05fv-fV85BA}{158.155.201.245}{158.155.201.245:9302}{dilm}{ml.machine_memory=16656986112, xpack.installed=true, ml.max_open_jobs=20}] which is not a quorum; discovery will continue using [158.155.72.11:9300] from hosts providers and [{node2}{npszcdMaSQOnpLvz8r1gjw}{8ukSXP07STm05fv-fV85BA}{158.155.201.245}{158.155.201.245:9302}{dilm}{ml.machine_memory=16656986112, xpack.installed=true, ml.max_open_jobs=20}] from last-known cluster state; node term 18, last-accepted version 3237 in term 17
[2020-06-26T06:42:10,506][TRACE][o.e.d.PeerFinder ] [node2] probing master nodes from cluster state: nodes:
{node2}{npszcdMaSQOnpLvz8r1gjw}{8ukSXP07STm05fv-fV85BA}{158.155.201.245}{158.155.201.245:9302}{dilm}{ml.machine_memory=16656986112, xpack.installed=true, ml.max_open_jobs=20}, local
[2020-06-26T06:42:10,507][TRACE][o.e.d.PeerFinder ] [node2] startProbe(158.155.201.245:9302) not probing local node
[2020-06-26T06:42:10,507][TRACE][o.e.d.SeedHostsResolver ] [node2] resolved host [158.155.201.245:9302] to [158.155.201.245:9302]
[2020-06-26T06:42:10,507][TRACE][o.e.d.SeedHostsResolver ] [node2] resolved host [158.155.72.11:9300] to [158.155.72.11:9300]
[2020-06-26T06:42:10,507][TRACE][o.e.d.PeerFinder ] [node2] probing resolved transport addresses [158.155.72.11:9300]
[2020-06-26T06:42:10,507][TRACE][o.e.d.PeerFinder ] [node2] Peer{transportAddress=158.155.72.11:9300, discoveryNode=null, peersRequestInFlight=false} attempting connection
[2020-06-26T06:42:10,508][TRACE][o.e.d.HandshakingTransportAddressConnector] [node2] [connectToRemoteMasterNode[158.155.72.11:9300]] opening probe connection
[2020-06-26T06:42:10,746][DEBUG][o.e.d.PeerFinder ] [node2] Peer{transportAddress=158.155.72.11:9300, discoveryNode=null, peersRequestInFlight=false} connection failed
org.elasticsearch.transport.ConnectTransportException: [][158.155.72.11:9300] general node connection failure
at org.elasticsearch.transport.TcpTransport$ChannelsConnectedListener.lambda$onResponse$2(TcpTransport.java:966) ~[elasticsearch-7.4.1.jar:7.4.1]
at org.elasticsearch.action.ActionListener$1.onFailure(ActionListener.java:70) ~[elasticsearch-7.4.1.jar:7.4.1]
at org.elasticsearch.transport.TransportHandshaker$HandshakeResponseHandler.handleLocalException(TransportHandshaker.java:155) ~[elasticsearch-7.4.1.jar:7.4.1]
at org.elasticsearch.transport.TransportHandshaker.lambda$sendHandshake$0(TransportHandshaker.java:67) ~[elasticsearch-7.4.1.jar:7.4.1]
at org.elasticsearch.action.ActionListener.lambda$wrap$0(ActionListener.java:131) ~[elasticsearch-7.4.1.jar:7.4.1]
at org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:62) ~[elasticsearch-7.4.1.jar:7.4.1]
at org.elasticsearch.action.ActionListener.lambda$toBiConsumer$3(ActionListener.java:159) ~[elasticsearch-7.4.1.jar:7.4.1]
at org.elasticsearch.common.concurrent.CompletableContext.lambda$addListener$0(CompletableContext.java:39) ~[elasticsearch-core-7.4.1.jar:7.4.1]
at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:859) ~[?:?]
at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:837) ~[?:?]
at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506) ~[?:?]
at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:2144) ~[?:?]
at org.elasticsearch.common.concurrent.CompletableContext.complete(CompletableContext.java:61) ~[elasticsearch-core-7.4.1.jar:7.4.1]
at org.elasticsearch.transport.netty4.Netty4TcpChannel.lambda$addListener$0(Netty4TcpChannel.java:61) ~[?:?]
at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:500) ~[?:?]
at io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:493) ~[?:?]
at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:472) ~[?:?]
at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:413) ~[?:?]
at io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:538) ~[?:?]
at io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:527) ~[?:?]
at io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:98) ~[?:?]
at io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) ~[?:?]
at io.netty.channel.AbstractChannel$CloseFuture.setClosed(AbstractChannel.java:1156) ~[?:?]
at io.netty.channel.AbstractChannel$AbstractUnsafe.doClose0(AbstractChannel.java:758) ~[?:?]
at io.netty.channel.AbstractChannel$AbstractUnsafe.close(AbstractChannel.java:734) ~[?:?]
at io.netty.channel.AbstractChannel$AbstractUnsafe.close(AbstractChannel.java:605) ~[?:?]
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.closeOnRead(AbstractNioByteChannel.java:105) ~[?:?]
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:171) ~[?:?]
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:697) ~[?:?]
at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:597) ~[?:?]
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:551) ~[?:?]
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:511) ~[?:?]
at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:918) ~[?:?]
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
at java.lang.Thread.run(Thread.java:830) [?:?]
Caused by: org.elasticsearch.transport.TransportException: handshake failed because connection reset
... 32 more
I am not using any ssl certificates , still i could see ssl handshake exception in the logs , also i tested the connectivity between two server and the ports where Elasticsearch services are hosted both server are reachable from each other. not understanding what's the problem here. Seeking your help
Regaeds ,
Ramya