Security/Basic Authentication/Transport TLS - One node failing No Subject Alternative names present

[hueyg]

Four node cluster, not sure if best setup but another time.

node1 - LS + ES, master eligible
node2 - LS + ES, master eligible
node3 - LS + ES, master eligible
node4 - KB + ES, ingest(?) only node

Following the security tutorial on encrypting traffic. Created CA, and used the ES cert util to do multiple on nodes one through three. Copied certs to the three nodes and updated elasticsearch.yml to enable security and point to these certs. Why did I forget node4? No idea. Just an idiot. Ran the cert util again to create the Node 4 cert.

Nodes one through three all start up fine concerning TLS and establish connection. Node 4 ES does not load:

[2019-08-26T16:25:30,080][WARN ][o.e.x.c.s.t.n.SecurityNetty4Transport] [usakibana] client did not trust this server's certificate, closing connection Netty4TcpChannel{localAddress=0.0.0.0/0.0.0.0:9300, remoteAddress=/192.168.25.19:56724
}

The other nodes show:

[2019-08-26T16:15:04,400][WARN ][o.e.t.TcpTransport       ] [usamon2] exception caught on transport layer [Netty4TcpChannel{localAddress=0.0.0.0/0.0.0.0:41298, remoteAddress=null}], closing connection
io.netty.handler.codec.DecoderException: javax.net.ssl.SSLHandshakeException: No subject alternative names present
        at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:472) ~[netty-codec-4.1.35.Final.jar:4.1.35.Final]
        at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:278) ~[netty-codec-4.1.35.Final.jar:4.1.35.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374) [netty-transport-4.1.35.Final.jar:4.1.35.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360) [netty-transport-4.1.35.Final.jar:4.1.35.Final]
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352) [netty-transport-4.1.35.Final.jar:4.1.35.Final]
        at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1408) [netty-transport-4.1.35.Final.jar:4.1.35.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374) [netty-transport-4.1.35.Final.jar:4.1.35.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360) [netty-transport-4.1.35.Final.jar:4.1.35.Final]
        at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:930) [netty-transport-4.1.35.Final.jar:4.1.35.Final]
        at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) [netty-transport-4.1.35.Final.jar:4.1.35.Final]
        at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:682) [netty-transport-4.1.35.Final.jar:4.1.35.Final]
        at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:582) [netty-transport-4.1.35.Final.jar:4.1.35.Final]
        at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:536) [netty-transport-4.1.35.Final.jar:4.1.35.Final]
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:496) [netty-transport-4.1.35.Final.jar:4.1.35.Final]
        at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:906) [netty-common-4.1.35.Final.jar:4.1.35.Final]
        at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) [netty-common-4.1.35.Final.jar:4.1.35.Final]
        at java.lang.Thread.run(Thread.java:835) [?:?]
Caused by: javax.net.ssl.SSLHandshakeException: No subject alternative names present

Here is sample of elasticsearch.yml:

xpack.security.enabled: true
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.keystore.path: config/certs/${node.name}.p12
xpack.security.transport.ssl.truststore.path: config/certs/${node.name}.p12

Here are the contents of my /config/certs:

-rw-------. 1 root elasticsearch 11112 Aug 26 16:23 certificate-bundle.zip
-rw-rw-rw-. 1 root elasticsearch  3443 Aug 26 16:00 node4.p12
drwxr-sr-x. 2 root elasticsearch    25 Aug  6 16:03 node1
drwxr-sr-x. 2 root elasticsearch    25 Aug  6 16:03 node2
drwxr-sr-x. 2 root elasticsearch    25 Aug  6 16:03 node3

[ikakavas (Ioannis Kakavas)]

Hi there,

See https://www.elastic.co/guide/en/elastic-stack-overview/7.3/trb-security-sslhandshake.html

What did you do differently when creating the node4.p12 that you didn't do for node 1 to 3 ? Did you use the same CA or did you maybe create a new one by mistake ?

[hueyg]

Thanks for the help Ioannis. As with most projects I am struggling with I did not take good enough notes to know whether I did anything different as I am only referencing the Elastic Stack documentation. However, it appears I did. Not sure exactly which step seems to have resolved the issue but going over it in case someone else encounters:

I viewed the p12 cert using:

openssl pkcs12 -info -in node4.p12

It seemed normal except for the friendlyName Attribute was set to "instance" instead of the node name/DNS name.

Based on your link I figured something was off so I copied over CA from a known working location and generated a new cert from scratch using the elasticsearch-certutil script.

I copied the new cert to my certs directory as specified in my eleasticsearch.yml file. Started ES and monitoring the cluster log I saw that this time it started and connected fine.

While I am not totally sure of a way to verify I have setup transport encryption, I am at least not receiving any errors. Now on to attempting to setup local authentication.

Thanks again Ioannis, you definitely pointed me in the right direction.