Elasticsearch can't restart and join the cluster

Elastic Stack Elasticsearch
1

Hello everyone, when I restart one node from Elasticsearch cluster, the node can't join cluster,I 'm sure I have config the security.I can't start my node, when I start it , my kibana can't login in with my old username and password.

cluster: 6 node
Machine：8C16G
ES version 7.12.1

Elasticsearch.yml config：

action.auto_create_index: .monitoring*,.watches,.triggered_watches,.watcher-history*,.ml*
#xpack.security.audit.enabled: true
xpack.security.enabled: true
xpack.security.transport.ssl.enabled: true
xpack.license.self_generated.type: basic
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.keystore.path: certs/elastic-certificates.p12
xpack.security.transport.ssl.truststore.path: certs/elastic-certificates.p12

ERROR Log:

[2021-11-08T20:19:25,102][WARN ][o.e.t.OutboundHandler ] [search3] send message failed [channel: Netty4TcpChannel{localAddress=/10.xx.xx.xx:17818, remoteAddress=/10.xx.xx.xxx:9200, profile=default}]
io.netty.handler.ssl.NotSslRecordException: not an SSL/TLS record: 485454502f312e30203430302042616420526571756573740d0a636f6e74656e742d747970653a206170706c69636174696f6e2f6a736f6e3b20636861727365743d5554462d380d0a636f6e74656e742d6c656e6774683a203432310d0a0d0a7b226572726f223a7b22726f6f745f6361757365223a5b7b2274797065223a22696c6c6567616c5f617267756d656e745f657863657074696f6e222c22726561736f6e223a22696e76616c69642076657273696f6e20666f726d61743a20255c75303030305c7530303141c294c2be47c38ec395c28337c38024c29150c39ec2a9c39ac2b4c398c2be4d4fc2b75324c293c388c2a2c29147c2914d5c7530303030285c75303031335c75303030325c75303031335c7530303031c3802cc3802bc38030c3802fc38024c38023c38028c38027c380227d5d2c2274797065223a22696c6c6567616c5f617267756d656e745f657863657074696f6e222c22726561736f6e223a22696e76616c69642076657273696f6e20666f726d61743a20255c75303030305c7530303141c294c2be47c38ec395c28337c38024c29150c39ec2a9c39ac2b4c398c2be4d4fc2b75324c293c388c2a2c29147c2914d5c7530303030285
at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1246) [netty-handler-4.1.49.Final.jar:4.1.49.Final]
at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1314) [netty-handler-4.1.49.Final.jar:4.1.49.Final]
at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:501) [netty-codec-4.1.49.Final.jar:4.1.49.Final]
at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:440) [netty-codec-4.1.49.Final.jar:4.1.49.Final]
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276) [netty-codec-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) [netty-common-4.1.49.Final.jar:4.1.49.Final]
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) [netty-common-4.1.49.Final.jar:4.1.49.Final]
at java.lang.Thread.run(Thread.java:831) [?:?]
2

It looks like this node it trying to connect to another node on 10.xx.xx.xxx:9200
9200 is usually the HTTP port, and nodes do not connect to one another on the HTTP port, they use the transport port (9300).

It looks like your seed nodes might be incorrectly configured.

1 Like
3

Yes, I config the seed nodes with 9200, when I modify to 9300, the node is ok, thank you very much :grinning: :+1:

4

Anyway, When I first set up the Elasticsearch cluster. I set the Elasticsearch.yml discovery.seed_hosts with http port 9200, My cluster set up success, It run success, Maybe Elasticsearch can give some error log message notfiy user modify the port to TCP port :grinning:
thank you

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.