Tips for failing handshake with shield

Hi,
I am trying to get SSL/TLS to work on my raspberry pi cluster. To bad I get
the following message:

[2015-02-11 17:51:35,037][ERROR][shield.transport.netty ] [jc-pi-red]
SSL/TLS handshake failed, closing channel: null

So I guess something is wrong in my certificate chain. I have created my
own CA and followed the steps from the getting started guide. Any tips on
how to start debugging?

thanks

Jettro

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/cf83e1d4-10b8-47b2-a5a0-c5c2491dc899%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

There should be another message on the other end of the connection that has
more details of the actual failure. That message can be seen on the client
side of the connection when the connection was closed by the server side.

Also, please
see Elasticsearch Platform — Find real-time answers at scale | Elastic
for common problems and some tips on how to resolve them.

On Wednesday, February 11, 2015 at 8:54:27 AM UTC-8, Jettro Coenradie wrote:

Hi,
I am trying to get SSL/TLS to work on my raspberry pi cluster. To bad I
get the following message:

[2015-02-11 17:51:35,037][ERROR][shield.transport.netty ] [jc-pi-red]
SSL/TLS handshake failed, closing channel: null

So I guess something is wrong in my certificate chain. I have created my
own CA and followed the steps from the getting started guide. Any tips on
how to start debugging?

thanks

Jettro

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/652d10f6-79ee-45a5-b919-eca3bc15a5d6%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Thanks Jay for your answer. Had a good look at the appendix with
troubleshooting, but cannot really find something to help me out. On the
client I see this message:

[2015-02-12 20:10:28,363][ERROR][shield.transport.netty ] [jc-pi-glas]
SSL/TLS handshake failed, closing channel: null

There is a stack trace after this error:
[2015-02-12 20:10:28,264][WARN ][shield.transport.netty ] [jc-pi-glas]
exception caught on transport layer [[id: 0x127799ce, /192.168.1.11:50409
=> 192.168.1.10/192.168.1.10:9300]], closing connection
java.lang.IllegalStateException: Internal error
at sun.security.ssl.SSLEngineImpl.initHandshaker(SSLEngineImpl.java:464)
at sun.security.ssl.SSLEngineImpl.readRecord(SSLEngineImpl.java:1001)
at sun.security.ssl.SSLEngineImpl.readNetRecord(SSLEngineImpl.java:901)
at sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:775)
at javax.net.ssl.SSLEngine.unwrap(SSLEngine.java:624)
at
org.elasticsearch.common.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1285)
at
org.elasticsearch.common.netty.handler.ssl.SslHandler.decode(SslHandler.java:917)
at
org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.jav

I also get numerous timeouts. The strange thing is that these timeout are
pings to the node itself.

[2015-02-12 20:10:25,151][WARN ][discovery.zen.ping.unicast] [jc-pi-glas]
failed to send ping to
[[jc-pi-glas][RBV7_u5yRsChF_M8Ep2ICQ][jc-pi-glas][inet[
192.168.1.11/192.168.1.11:9300]]]
org.elasticsearch.transport.ReceiveTimeoutTransportException:
[jc-pi-glas][inet[
192.168.1.11/192.168.1.11:9300]][internal:discovery/zen/unicast] request_id
[5] timed out after [3751ms]
at
org.elasticsearch.transport.TransportService$TimeoutHandler.run(TransportService.java:366)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:744)

On the red pi I get this message:
[2015-02-12 20:10:44,245][WARN ][shield.transport.netty ] [jc-pi-red]
exception caught on transport layer [[id: 0xdc4d94e1, /192.168.1.11:50409
=> /192.168.1.10:9300]], closing connection
javax.net.ssl.SSLException: Received close_notify during handshake
at sun.security.ssl.Alerts.getSSLException(Alerts.java:208)
at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1646)
at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1614)

I also created a very small java application with sockets and used the same
jks files to authenticate. There it works.

Any other ideas?

On Wed, Feb 11, 2015 at 7:05 PM, Jay Modi jay.modi@elasticsearch.com
wrote:

There should be another message on the other end of the connection that
has more details of the actual failure. That message can be seen on the
client side of the connection when the connection was closed by the server
side.

Also, please see
Elasticsearch Platform — Find real-time answers at scale | Elastic
for common problems and some tips on how to resolve them.

On Wednesday, February 11, 2015 at 8:54:27 AM UTC-8, Jettro Coenradie
wrote:

Hi,
I am trying to get SSL/TLS to work on my raspberry pi cluster. To bad I
get the following message:

[2015-02-11 17:51:35,037][ERROR][shield.transport.netty ] [jc-pi-red]
SSL/TLS handshake failed, closing channel: null

So I guess something is wrong in my certificate chain. I have created my
own CA and followed the steps from the getting started guide. Any tips on
how to start debugging?

thanks

Jettro

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/g-AT4CAVBCw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/652d10f6-79ee-45a5-b919-eca3bc15a5d6%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/652d10f6-79ee-45a5-b919-eca3bc15a5d6%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
Jettro Coenradie

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CABB4caMUbzCHTUSn0VcQfiep24nMpYnweJ796_s4N0bbc8F%2Biw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

I think you are running into the TLS handshake taking too long and it
causes the pings to timeout and then the Netty channels are closed, which
is why you see the connection closed messages. Since you are running on a
Raspberry PI, the overhead of TLS is going to be a lot for that platform;
the handshake is expensive in terms of CPU. You probably need to increase
the zen ping timeout and based on your other post it may need to be like
over 25 seconds.

On Thursday, February 12, 2015 at 11:20:35 AM UTC-8, Jettro Coenradie wrote:

Thanks Jay for your answer. Had a good look at the appendix with
troubleshooting, but cannot really find something to help me out. On the
client I see this message:

[2015-02-12 20:10:28,363][ERROR][shield.transport.netty ] [jc-pi-glas]
SSL/TLS handshake failed, closing channel: null

There is a stack trace after this error:
[2015-02-12 20:10:28,264][WARN ][shield.transport.netty ] [jc-pi-glas]
exception caught on transport layer [[id: 0x127799ce, /192.168.1.11:50409
=> 192.168.1.10/192.168.1.10:9300]
http://192.168.1.10/192.168.1.10:9300]], closing connection
java.lang.IllegalStateException: Internal error
at sun.security.ssl.SSLEngineImpl.initHandshaker(SSLEngineImpl.java:464)
at sun.security.ssl.SSLEngineImpl.readRecord(SSLEngineImpl.java:1001)
at sun.security.ssl.SSLEngineImpl.readNetRecord(SSLEngineImpl.java:901)
at sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:775)
at javax.net.ssl.SSLEngine.unwrap(SSLEngine.java:624)
at
org.elasticsearch.common.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1285)
at
org.elasticsearch.common.netty.handler.ssl.SslHandler.decode(SslHandler.java:917)
at
org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.jav

I also get numerous timeouts. The strange thing is that these timeout are
pings to the node itself.

[2015-02-12 20:10:25,151][WARN ][discovery.zen.ping.unicast] [jc-pi-glas]
failed to send ping to
[[jc-pi-glas][RBV7_u5yRsChF_M8Ep2ICQ][jc-pi-glas][inet[
192.168.1.11/192.168.1.11:9300] http://192.168.1.11/192.168.1.11:9300]
]]
org.elasticsearch.transport.ReceiveTimeoutTransportException:
[jc-pi-glas][inet[
192.168.1.11/192.168.1.11:9300]][internal:discovery/zen/unicast
http://192.168.1.11/192.168.1.11:9300]][internal:discovery/zen/unicast]
request_id [5] timed out after [3751ms]
at
org.elasticsearch.transport.TransportService$TimeoutHandler.run(TransportService.java:366)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:744)

On the red pi I get this message:
[2015-02-12 20:10:44,245][WARN ][shield.transport.netty ] [jc-pi-red]
exception caught on transport layer [[id: 0xdc4d94e1, /192.168.1.11:50409
=> /192.168.1.10:9300]], closing connection
javax.net.ssl.SSLException: Received close_notify during handshake
at sun.security.ssl.Alerts.getSSLException(Alerts.java:208)
at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1646)
at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1614)

I also created a very small java application with sockets and used the
same jks files to authenticate. There it works.

Any other ideas?

On Wed, Feb 11, 2015 at 7:05 PM, Jay Modi <jay....@elasticsearch.com
<javascript:>> wrote:

There should be another message on the other end of the connection that
has more details of the actual failure. That message can be seen on the
client side of the connection when the connection was closed by the server
side.

Also, please see
Elasticsearch Platform — Find real-time answers at scale | Elastic
for common problems and some tips on how to resolve them.

On Wednesday, February 11, 2015 at 8:54:27 AM UTC-8, Jettro Coenradie
wrote:

Hi,
I am trying to get SSL/TLS to work on my raspberry pi cluster. To bad I
get the following message:

[2015-02-11 17:51:35,037][ERROR][shield.transport.netty ] [jc-pi-red]
SSL/TLS handshake failed, closing channel: null

So I guess something is wrong in my certificate chain. I have created my
own CA and followed the steps from the getting started guide. Any tips on
how to start debugging?

thanks

Jettro

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/g-AT4CAVBCw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/652d10f6-79ee-45a5-b919-eca3bc15a5d6%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/652d10f6-79ee-45a5-b919-eca3bc15a5d6%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
Jettro Coenradie
http://www.gridshore.nl

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/7308bc30-f3bc-4634-b292-bac9615a259f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Hi Jay, thanks again for the help. You were so right. Change the timeout to
30s and now the cluster is up and running. Now I can go for the next steps.

regards Jettro

On Fri, Feb 13, 2015 at 3:12 AM, Jay Modi jay.modi@elasticsearch.com
wrote:

I think you are running into the TLS handshake taking too long and it
causes the pings to timeout and then the Netty channels are closed, which
is why you see the connection closed messages. Since you are running on a
Raspberry PI, the overhead of TLS is going to be a lot for that platform;
the handshake is expensive in terms of CPU. You probably need to increase
the zen ping timeout and based on your other post it may need to be like
over 25 seconds.

On Thursday, February 12, 2015 at 11:20:35 AM UTC-8, Jettro Coenradie
wrote:

Thanks Jay for your answer. Had a good look at the appendix with
troubleshooting, but cannot really find something to help me out. On the
client I see this message:

[2015-02-12 20:10:28,363][ERROR][shield.transport.netty ] [jc-pi-glas]
SSL/TLS handshake failed, closing channel: null

There is a stack trace after this error:
[2015-02-12 20:10:28,264][WARN ][shield.transport.netty ] [jc-pi-glas]
exception caught on transport layer [[id: 0x127799ce, /192.168.1.11:50409
=> 192.168.1.10/192.168.1.10:9300]
http://192.168.1.10/192.168.1.10:9300]], closing connection
java.lang.IllegalStateException: Internal error
at sun.security.ssl.SSLEngineImpl.initHandshaker(SSLEngineImpl.java:464)
at sun.security.ssl.SSLEngineImpl.readRecord(SSLEngineImpl.java:1001)
at sun.security.ssl.SSLEngineImpl.readNetRecord(SSLEngineImpl.java:901)
at sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:775)
at javax.net.ssl.SSLEngine.unwrap(SSLEngine.java:624)
at org.elasticsearch.common.netty.handler.ssl.SslHandler.
unwrap(SslHandler.java:1285)
at org.elasticsearch.common.netty.handler.ssl.SslHandler.
decode(SslHandler.java:917)
at org.elasticsearch.common.netty.handler.codec.frame.
FrameDecoder.callDecode(FrameDecoder.jav

I also get numerous timeouts. The strange thing is that these timeout are
pings to the node itself.

[2015-02-12 20:10:25,151][WARN ][discovery.zen.ping.unicast] [jc-pi-glas]
failed to send ping to [[jc-pi-glas][RBV7_u5yRsChF_
M8Ep2ICQ][jc-pi-glas][inet[192.168.1.11/192.168.1.11:9300]
http://192.168.1.11/192.168.1.11:9300]]]
org.elasticsearch.transport.ReceiveTimeoutTransportException:
[jc-pi-glas][inet[192.168.1.11/192.168.1.11:9300]][
internal:discovery/zen/unicast
http://192.168.1.11/192.168.1.11:9300]][internal:discovery/zen/unicast]
request_id [5] timed out after [3751ms]
at org.elasticsearch.transport.TransportService$TimeoutHandler.run(
TransportService.java:366)
at java.util.concurrent.ThreadPoolExecutor.runWorker(
ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(
ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:744)

On the red pi I get this message:
[2015-02-12 20:10:44,245][WARN ][shield.transport.netty ] [jc-pi-red]
exception caught on transport layer [[id: 0xdc4d94e1, /192.168.1.11:50409
=> /192.168.1.10:9300]], closing connection
javax.net.ssl.SSLException: Received close_notify during handshake
at sun.security.ssl.Alerts.getSSLException(Alerts.java:208)
at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1646)
at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1614)

I also created a very small java application with sockets and used the
same jks files to authenticate. There it works.

Any other ideas?

On Wed, Feb 11, 2015 at 7:05 PM, Jay Modi jay....@elasticsearch.com
wrote:

There should be another message on the other end of the connection that
has more details of the actual failure. That message can be seen on the
client side of the connection when the connection was closed by the server
side.

Also, please see Elasticsearch Platform — Find real-time answers at scale | Elastic
trouble-shooting.html#_sslhandshakeexception_causing_connections_to_fail
for common problems and some tips on how to resolve them.

On Wednesday, February 11, 2015 at 8:54:27 AM UTC-8, Jettro Coenradie
wrote:

Hi,
I am trying to get SSL/TLS to work on my raspberry pi cluster. To bad I
get the following message:

[2015-02-11 17:51:35,037][ERROR][shield.transport.netty ]
[jc-pi-red] SSL/TLS handshake failed, closing channel: null

So I guess something is wrong in my certificate chain. I have created
my own CA and followed the steps from the getting started guide. Any tips
on how to start debugging?

thanks

Jettro

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/
topic/elasticsearch/g-AT4CAVBCw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/652d10f6-79ee-45a5-b919-eca3bc15a5d6%
40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/652d10f6-79ee-45a5-b919-eca3bc15a5d6%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
Jettro Coenradie
http://www.gridshore.nl

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/g-AT4CAVBCw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/7308bc30-f3bc-4634-b292-bac9615a259f%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/7308bc30-f3bc-4634-b292-bac9615a259f%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
Jettro Coenradie

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CABB4caN3PUaUBi6sVh9j9hDM7xyqxGwjPw3Sb7iW4DLUJsjRRw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.