Hi, I have a huge problem w Elasticsearch cross cluster search. I have two clusters that have the same version and licensing levels. They are both signed by the same CA. I am trying to configure cross cluster search between two clusters and the error i still get is following:
"log.level": "WARN", "message":"exception caught on transport layer
[Netty4TcpChannel{localAddress=/****:56246, remoteAddress=elasticsearch-master.elastic/****:31155, profile=default}], closing connection", "ecs.version":
"1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","proce
ss.thread.name":"elasticsearch[***][transport_worker]
[T#24]","log.logger":"org.elasticsearch.transport.TcpTransport","elasticsearch
.cluster.uuid":"tqWk7LChSCSxQ3RNXJGP9g","elasticsearch.node.id":"aNy4D0
nNRwmEwQE8lH8D1g","elasticsearch.node.name":"***","elasticsearch.
cluster.name":"****","error.type":"io.netty.handler.codec.DecoderException"
,"error.message":"javax.net.ssl.SSLHandshakeException:
(certificate_required) Received fatal alert:
certificate_required","error.stack_trace":"io.netty.handler.codec.DecoderExce
ption: javax.net.ssl.SSLHandshakeException: (certificate_required) Received
fatal alert: certificate_required\n\tat io.netty.codec@4.1.115.Final/io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageD
ecoder.java:500)\n\tat io.netty.codec@4.1.115.Final/
io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessage
Decoder.java:290)\n\tat io.netty.transport@4.1.115.Final/
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(Abstra
ctChannelHandlerContext.java:444)\n\tat io.netty.transport@4.1.115.Final/
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(Abstra
ctChannelHandlerContext.java:420)\n\tat io.netty.transport@4.1.115.Final/
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractC
hannelHandlerContext.java:412)\n\tat io.netty.transport@4.1.115.Final/
io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(Default
ChannelPipeline.java:1357)\n\tat io.netty.transport@4.1.115.Final/
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440)\n\tat io.netty.transport@4.1.115.Final/
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(Abstra
ctChannelHandlerContext.java:420)\n\tat io.netty.transport@4.1.115.Final/
io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPip
eline.java:868)\n\tat io.netty.transport@4.1.115.Final/
io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractN
ioByteChannel.java:166)\n\tat io.netty.transport@4.1.115.Final/
io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:7
88)\n\tat io.netty.transport@4.1.115.Final/
io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.j
ava:689)\n\tat io.netty.transport@4.1.115.Final/
io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:6
52)\n\tat io.netty.transport@4.1.115.Final/
io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562)\n\tat
io.netty.common@4.1.115.Final/
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEvent
Executor.java:997)\n\tat io.netty.common@4.1.115.Final/
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
\n\tat java.base/java.lang.Thread.run(Thread.java:1575)\nCaused by:
javax.net.ssl.SSLHandshakeException: (certificate_required) Received fatal
alert: certificate_required\n\tat java.base/
sun.security.ssl.Alert.createSSLException(Alert.java:130)\n\tat java.base/sun.security.ssl.Alert.createSSLException(Alert.java:117)\n\tat java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:365)\n\tat java.base/sun.security.ssl.Alert$AlertConsumer.consume(Alert.java:287)\n\tat java.base/sun.security.ssl.TransportContext.dispatch(TransportContext.java:204)\n\tat java.base/sun.security.ssl.SSLTransport.decode(SSLTransport.java:172)\n\tat java.base/sun.security.ssl.SSLEngineImpl.decode(SSLEngineImpl.java:736)\n\tat java.base/sun.security.ssl.SSLEngineImpl.readRecord(SSLEngineImpl.java:691)\n\tat java.base/sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:506)\n\tat java.base/sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:482)\n\tat java.base/javax.net.ssl.SSLEngine.unwrap(SSLEngine.java:679)\n\tat io.netty.handler@4.1.115.Final/io.netty.handler.ssl.SslHandler$SslEngineType$3.unwrap(SslHandler.java:309)\n\tat io.netty.handler@4.1.115.Final/io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1473)\n\tat io.netty.handler@4.1.115.Final/io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1366)\n\tat io.netty.handler@4.1.115.Final/io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1415)\n\tat io.netty.codec@4.1.115.Final/io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:530)\n\tat io.netty.codec@4.1.115.Final/io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:469)\n\t... 16 more\n"}
and especially this part:
nNRwmEwQE8lH8D1g","elasticsearch.node.name":"***","elasticsearch.
cluster.name":"****","error.type":"io.netty.handler.codec.DecoderException"
,"error.message":"javax.net.ssl.SSLHandshakeException:
(certificate_required) Received fatal alert:
certificate_required","error.stack_trace":"io.netty.handler.codec.DecoderExce
ption: javax.net.ssl.SSLHandshakeException: (certificate_required) Received
fatal alert: certificate_required\n
I know it is a two way TLS authentication so it is logical that I need a server instance certificate. But where should I put it? In Java trust store? Does the remote cluster need my certificate in its trust store? What is the proper way to address this?
What certificate is required? The documentation states that I only need both ca to be trusted by one another and this is already configured. Can someone please give me any hint?