# Unable to use es-hadoop with TLS enabled on my ES cluster
**URL:** https://discuss.elastic.co/t/unable-to-use-es-hadoop-with-tls-enabled-on-my-es-cluster/197225
**Category:** Elasticsearch
**Tags:** es-hadoop
**Created:** [August 28, 2019, 8:55pm UTC](https://discuss.elastic.co/t/unable-to-use-es-hadoop-with-tls-enabled-on-my-es-cluster/197225 "2019-08-28T20:55:27Z")
**Posts on this page:** 3
**Page:** 1
### Author:  [@mstarensier](https://discuss.elastic.co/u/mstarensier)
#### Post date: [August 28, 2019, 8:55pm UTC](https://discuss.elastic.co/t/unable-to-use-es-hadoop-with-tls-enabled-on-my-es-cluster/197225/1 "2019-08-28T20:55:27Z")
I'm having trouble configuring TLS for es-hadoop.
Running 7.2 Elasticsearch in a 3 node cluster (1 data, 1 master, 1 coordinating which hosts kibana).
Running Spark on an EMR cluster in the same vpn. Before enabling xpack security, was able to connect to ES from Spark using both es-hadoop and Rest Client.
Configured TLS with CA and single signing cert, generated by certutil, used in all es and spark nodes.
The nodes all ES nodes communicate with each other correctly.
But in spark, the es-hadoop connector can no longer connect to the ES cluster. (The rest client running in the spark job can connect using basic auth). For experimentation, we are running this job from a zeppelin notebook spark interpreter.
I am using the same certificate in the spark cluster nodes which I am using in the ES cluster nodes.
spark es config properties are:
es.net.http.auth.pass : xxxxxx
es.net.http.auth.user : xxxxxx
es.net.ssl : true
es.net.ssl.keystore.location : file:///home/hadoop/ria/certs/test-es-node-cert.p12
es.net.ssl.trustore.location : file:///home/hadoop/ria/certs/test-es-node-cert.p12
es.nodes : 10.0.1.179
es.port : 9300
Rest client works fine, connector fails. In the spark logs on failure, message begins:
%text org.elasticsearch.hadoop.EsHadoopIllegalArgumentExc eption: Cannot detect ES version - typically this happens if the network/Elasticsearch cluster is not accessible or when targeting a WAN/Cloud instance without the proper setting 'es.nodes.wan.only' at org.elasticsearch.hadoop.rest.InitializationUtils.discoverClusterInfo(InitializationUtils.java:340) at org.elasticsearch.hadoop.rest.RestService.findPartitions(RestService.java:220) at
in es logs, stack trace is
exception caught on transport layer [Netty4TcpChannel{localAddress=0.0.0.0/0.0.0.0:9300, remoteAddress=/10.0.1.81:51709}], closing connection io.netty.handler.codec.DecoderException: javax.net.ssl.SSLException: Received fatal alert: internal\_error at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:472) ~[netty-codec-4.1.35.Final.jar:4.1.35.Final] at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:278) ~[netty-codec-4.1.35.Final.jar:4.1.35.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374) [netty-transport-4.1.35.Final.jar:4.1.35.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360) [netty-transport-4.1.35.Final.jar:4.1.35.Final] at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352) [netty-transport-4.1.35.Final.jar:4.1.35.Final] at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1408) [netty-transport-4.1.35.Final.jar:4.1.35.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374) [netty-transport-4.1.35.Final.jar:4.1.35.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360) [netty-transport-4.1.35.Final.jar:4.1.35.Final] at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:930) [netty-transport-4.1.35.Final.jar:4.1.35.Final] at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) [netty-transport-4.1.35.Final.jar:4.1.35.Final] at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:682) [netty-transport-4.1.35.Final.jar:4.1.35.Final] at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:582) [netty-transport-4.1.35.Final.jar:4.1.35.Final] at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:536) [netty-transport-4.1.35.Final.jar:4.1.35.Final] at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:496) [netty-transport-4.1.35.Final.jar:4.1.35.Final] at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:906) [netty-common-4.1.35.Final.jar:4.1.35.Final] at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) [netty-common-4.1.35.Final.jar:4.1.35.Final] at java.lang.Thread.run(Thread.java:835) [?:?] Caused by: javax.net.ssl.SSLException: Received fatal alert: internal\_error at sun.security.ssl.Alert.createSSLException(Alert.java:133) ~[?:?] at sun.security.ssl.Alert.createSSLException(Alert.java:117) ~[?:?] at sun.security.ssl.TransportContext.fatal(TransportContext.java:307) ~[?:?] at sun.security.ssl.Alert$AlertConsumer.consume(Alert.java:285) ~[?:?] at sun.security.ssl.TransportContext.dispatch(TransportContext.java:180) ~[?:?] at sun.security.ssl.SSLTransport.decode(SSLTransport.java:164) ~[?:?] at sun.security.ssl.SSLEngineImpl.decode(SSLEngineImpl.java:681) ~[?:?] at sun.security.ssl.SSLEngineImpl.readRecord(SSLEngineImpl.java:636) ~[?:?] at sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:454) ~[?:?] at sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:433) ~[?:?] at javax.net.ssl.SSLEngine.unwrap(SSLEngine.java:634) ~[?:?] at io.netty.handler.ssl.SslHandler$SslEngineType$3.unwrap(SslHandler.java:295) ~[netty-handler-4.1.35.Final.jar:4.1.35.Final] at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1332) ~[netty-handler-4.1.35.Final.jar:4.1.35.Final] at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1227) ~[netty-handler-4.1.35.Final.jar:4.1.35.Final] at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1274) ~[netty-handler-4.1.35.Final.jar:4.1.35.Final] at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:502) ~[netty-codec-4.1.35.Final.jar:4.1.35.Final] at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:441) ~[netty-codec-4.1.35.Final.jar:4.1.35.Final] ... 16 more [2019-08-28T14:12:16,592][WARN][o.e.t.TcpTransport] [test-es-data-node1] exception caught on transport layer [Netty4TcpChannel{localAddress=0.0.0.0/0.0.0.0:9300, remoteAddress=/10.0.1.81:39683}], closing connection io.netty.handler.codec.DecoderException: javax.net.ssl.SSLException: Received fatal alert: internal\_error at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:472) ~[netty-codec-4.1.35.Final.jar:4.1.35.Final] at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:278) ~[netty-codec-4.1.35.Final.jar:4.1.35.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374) [netty-transport-4.1.35.Final.jar:4.1.35.Final] at 4
elided....
Are there some other additional or different configurations required? Do I need a different certificate to act as the es-hadoop keystore? Any ideas would be helpful.
---
### Author:  [@james.baiera](https://discuss.elastic.co/u/james.baiera)
#### Post date: [September 6, 2019, 4:16pm UTC](https://discuss.elastic.co/t/unable-to-use-es-hadoop-with-tls-enabled-on-my-es-cluster/197225/2 "2019-09-06T16:16:38Z")
You may need to set the keystore type as detailed in the [SSL settings](https://www.elastic.co/guide/en/elasticsearch/hadoop/current/configuration.html#_ssl) for ES-Hadoop. Outside of that, I would increase the logging level to `TRACE` in the `org.elasticsearch.hadoop.rest.commonshttp` package to see what the network layer of ES-Hadoop is trying to do. Hopefully it highlights the full error information
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [October 4, 2019, 4:16pm UTC](https://discuss.elastic.co/t/unable-to-use-es-hadoop-with-tls-enabled-on-my-es-cluster/197225/3 "2019-10-04T16:16:49Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.