After upgrading from elastic-search 5.4.0 -> 5.6.8 -> 6.2.3 it is necessary to add TLS to the elastic-search cluster. I attempted to add TLS configuration to a single node cluster operating in production mode (for test purposes only), but am still planning on access from HTTP as all my current clients are trusted and behind the firewall.
My configuration around transport is:
xpack.security.enabled: true
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.keystore.path : /etc/elasticsearch/elastic-certificates.p12
xpack.security.transport.ssl.truststore.path : /etc/elasticsearch/elastic-certificates.p12
xpack.security.http.ssl.enabled: false
The exceptions in the logs that I receive on elastic-search start-up are as follows:
[2018-04-09T15:11:29,352][INFO ][o.e.t.TransportService ] [zMOSE-6] publish_address {172.18.0.1:9300}, bound_addresses {[::]:9300}
[2018-04-09T15:11:29,533][INFO ][o.e.b.BootstrapChecks ] [zMOSE-6] bound or publishing to a non-loopback address, enforcing bootstrap checks
[2018-04-09T15:11:33,137][INFO ][o.e.c.s.MasterService ] [zMOSE-6] zen-disco-elected-as-master ([0] nodes joined), reason: new_master {zMOSE-6}{zMOSE-6fQ3m56HUYL5_lZA}{4DMWtkRJRLisr4_6vggvbQ}{172.18.0.1}{172.18.0.1:9300}
[2018-04-09T15:11:33,185][INFO ][o.e.c.s.ClusterApplierService] [zMOSE-6] new_master {zMOSE-6}{zMOSE-6fQ3m56HUYL5_lZA}{4DMWtkRJRLisr4_6vggvbQ}{172.18.0.1}{172.18.0.1:9300}, reason: apply cluster state (from master [master {zMOSE-6}{zMOSE-6fQ3m56HUYL5_lZA}{4DMWtkRJRLisr4_6vggvbQ}{172.18.0.1}{172.18.0.1:9300} committed version [1] source [zen-disco-elected-as-master ([0] nodes joined)]])
[2018-04-09T15:11:35,967][WARN ][o.e.x.s.t.n.SecurityNetty4ServerTransport] [zMOSE-6] exception caught on transport layer [NettyTcpChannel{localAddress=0.0.0.0/0.0.0.0:9300, remoteAddress=/0:0:0:0:0:0:0:1:48872}], closing connection
io.netty.handler.codec.DecoderException: javax.net.ssl.SSLException: Received close_notify during handshake
at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:459) ~[netty-codec-4.1.16.Final.jar:4.1.16.Final]
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:265) ~[netty-codec-4.1.16.Final.jar:4.1.16.Final]
at
Caused by: javax.net.ssl.SSLException: Received close_notify during handshake
at sun.security.ssl.Alerts.getSSLException(Alerts.java:208) ~[?:?]
at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1666) ~[?:?]
at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1634) ~[?:?]
at sun.security.ssl.SSLEngineImpl.recvAlert(SSLEngineImpl.java:1776) ~[?:?]
at sun.security.ssl.SSLEngineImpl.readRecord(SSLEngineImpl.java:1083) ~[?:?]
at sun.security.ssl.SSLEngineImpl.readNetRecord(SSLEngineImpl.java:907) ~[?:?]
at sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:781) ~[?:?]
at javax.net.ssl.SSLEngine.unwrap(SSLEngine.java:624) ~[?:1.8.0_161]
at io.netty.handler.ssl.SslHandler$SslEngineType$3.unwrap(SslHandler.java:281) ~[?:?]
at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1215) ~[?:?]
at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1127) ~[?:?]
at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1162) ~[?:?]
at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:489) ~[?:?]
at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:428) ~[?:?]
[2018-04-09T15:11:36,282][INFO ][o.e.x.s.t.n.SecurityNetty4HttpServerTransport] [zMOSE-6] publish_address {172.18.0.1:9200}, bound_addresses {[::]:9200}
[2018-04-09T15:11:36,283][INFO ][o.e.n.Node ] [zMOSE-6] started
[2018-04-09T15:11:38,896][ERROR][o.e.x.m.c.i.IndexRecoveryCollector] [zMOSE-6] collector [index_recovery] failed to collect data
org.elasticsearch.cluster.block.ClusterBlockException: blocked by: [SERVICE_UNAVAILABLE/1/state not recovered / initialized];
...
[2018-04-09T15:11:39,978][ERROR][o.e.x.m.c.c.ClusterStatsCollector] [zMOSE-6] collector [cluster_stats] failed to collect data
java.lang.NullPointerException: null
at org.elasticsearch.xpack.monitoring.collector.cluster.ClusterStatsCollector.doCollect(ClusterStatsCollector.java:119) ~[x-pack-monitoring-6.2.3.jar:6.2.3]
at org.elasticsearch.xpack.monitoring.collector.Collector.collect(Collector.java:99) [x-pack-monitoring-6.2.3.jar:6.2.3]
at org.elasticsearch.xpack.monitoring.MonitoringService$MonitoringExecution$1.doRun(MonitoringService.java:221) [x-pack-monitoring-6.2.3.jar:6.2.3]
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-6.2.3.jar:6.2.3]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_161]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_161]
at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:573) [elasticsearch-6.2.3.jar:6.2.3]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_161]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_161]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_161]
...
Can you provide some guidance on the reason for these error?