# X-pack installation to ELK stack

**URL:** <https://discuss.elastic.co/t/x-pack-installation-to-elk-stack/113944>\
**Category:** Elasticsearch\
**Created:** [January 3, 2018, 3:00pm UTC](https://discuss.elastic.co/t/x-pack-installation-to-elk-stack/113944 "2018-01-03T15:00:57Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![SJN8](https://avatars.discourse-cdn.com/v4/letter/s/f08c70/32.png) [@SJN8](https://discuss.elastic.co/u/SJN8)\
**Post date:** [January 3, 2018, 3:00pm UTC](https://discuss.elastic.co/t/x-pack-installation-to-elk-stack/113944/1 "2018-01-03T15:00:57Z")

</div>

Hi All ,

i have a working ELK stack with Filebeat and kafka .

I want to secure it with X-pack .  
I have gone through the documents but i am facing issue while implementing it .

Can someone help on this ?

i can share details if required .

---

<div class="post-metadata">

**Author:** ![TimV](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/timv/32/13162_2.png) [@TimV](https://discuss.elastic.co/u/TimV)\
**Post date:** [January 3, 2018, 5:44pm UTC](https://discuss.elastic.co/t/x-pack-installation-to-elk-stack/113944/2 "2018-01-03T17:44:09Z")

</div>

If you have a commercial subscription with us, then your support engineer can help.

Otherwise, we're able to support you here, but you will need to provide much more information.

- What version are you running?
- What steps have you taken?
- What issue did you run into?
- What have you tried so far?

---

<div class="post-metadata">

**Author:** ![SJN8](https://avatars.discourse-cdn.com/v4/letter/s/f08c70/32.png) [@SJN8](https://discuss.elastic.co/u/SJN8)\
**Post date:** [January 4, 2018, 6:37am UTC](https://discuss.elastic.co/t/x-pack-installation-to-elk-stack/113944/3 "2018-01-04T06:37:03Z")

</div>

I dont have commercial subsciption , i am testing with trial version .

Below are the details ,

- i am testing with 6.1.1 versions of Elastic search and X-pack .

- I have installed x-pack on all 3 nodes of elastic search cluster . also generated node certificate for all 3 nodes .

- Update elasticsearch.yml with below ,

###################################################### X-Pack ###############################################  
action.auto\_create\_index: .security,.monitoring\*,.watches,.triggered\_watches,.watcher-history\*,.ml,\*  
xpack.ssl.key: /etc/elasticsearch/certs/node01/node01.key  
xpack.ssl.certificate: /etc/elasticsearch/certs/node01/node01.crt  
xpack.ssl.certificate\_authorities: /etc/elasticsearch/certs/node01/node01.crt  
xpack.security.transport.ssl.enabled: true  
xpack.security.http.ssl.enabled: true

on all 3 nodes .

I have generated certificates on 1 node for all 3 nodes with certgen cmd .

- Now after generating certificates and copying them to each node and updating yml . when i am restarting elastic cluster , it is throwing below exceptions ,

exceptions are related to certificates ,

[2018-01-04T06:22:13,397][WARN][o.e.x.s.t.n.SecurityNetty4Transport] [node02] send message failed [channel: org.  
elasticsearch.transport.netty4.NettyTcpChannel@de4289b]  
javax.net.ssl.SSLException: Received fatal alert: bad\_certificate  
at sun.security.ssl.Alerts.getSSLException(Alerts.java:208) ~[?:?]  
at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1666) ~[?:?]  
at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1634) ~[?:?]  
at sun.security.ssl.SSLEngineImpl.recvAlert(SSLEngineImpl.java:1800) ~[?:?]  
at sun.security.ssl.SSLEngineImpl.readRecord(SSLEngineImpl.java:1083) ~[?:?]  
at sun.security.ssl.SSLEngineImpl.readNetRecord(SSLEngineImpl.java:907) ~[?:?]  
at sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:781) ~[?:?]  
at javax.net.ssl.SSLEngine.unwrap(SSLEngine.java:624) ~[?:1.8.0\_151]  
at io.netty.handler.ssl.SslHandler$SslEngineType$3.unwrap(SslHandler.java:255) ~[netty-handler-4.1.13.Final.jar:4.1.13.Fin  
al]  
at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1162) [netty-handler-4.1.13.Final.jar:4.1.13.Final]  
at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1084) [netty-handler-4.1.13.Final.jar:4.1.13.Final]  
at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:489) [netty-codec-  
4.1.13.Final.jar:4.1.13.Final]  
at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:428) [netty-codec-4.1.13.Final.jar:4.1  
.13.Final]  
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:265) [netty-codec-4.1.13.Final.jar:4.  
1.13.Final]  
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transpo  
rt-4.1.13.Final.jar:4.1.13.Final]  
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transpo  
rt-4.1.13.Final.jar:4.1.13.Final]  
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport  
-4.1.13.Final.jar:4.1.13.Final]  
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1334) [netty-transport-4.1.  
13.Final.jar:4.1.13.Final]  
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transpo  
rt-4.1.13.Final.jar:4.1.13.Final]  
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transpo  
rt-4.1.13.Final.jar:4.1.13.Final]  
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:926) [netty-transport-4.1.13.Final.  
jar:4.1.13.Final]  
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:134) [netty-transport-4.1.13  
.Final.jar:4.1.13.Final]  
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:644) [netty-transport-4.1.13.Final.jar:4.1.13.Fi  
nal]  
at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:544) [netty-transport-4.1.13.Final.jar:4.1  
.13.Final]  
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:498) [netty-transport-4.1.13.Final.jar:4.1.13.F  
inal]  
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:458) [netty-transport-4.1.13.Final.jar:4.1.13.Final]  
at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858) [netty-common-4.1.13.Final  
.jar:4.1.13.Final]  
at java.lang.Thread.run(Thread.java:748) [?:1.8.0\_151]  
[2018-01-04T06:22:13,399][WARN][o.e.x.s.t.n.SecurityNetty4Transport] [node02] exception caught on transport laye  
r [org.elasticsearch.transport.netty4.NettyTcpChannel@de4289b], closing connection  
io.netty.handler.codec.DecoderException: javax.net.ssl.SSLException: Received fatal alert: bad\_certificate

---

<div class="post-metadata">

**Author:** ![SJN8](https://avatars.discourse-cdn.com/v4/letter/s/f08c70/32.png) [@SJN8](https://discuss.elastic.co/u/SJN8)\
**Post date:** [January 4, 2018, 6:39am UTC](https://discuss.elastic.co/t/x-pack-installation-to-elk-stack/113944/4 "2018-01-04T06:39:38Z")

</div>

exception continued ,

at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:459) ~[netty-codec-4.1.13.Final.jar:4.  
1.13.Final]  
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:265) ~[netty-codec-4.1.13.Final.jar:4  
.1.13.Final]  
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transpo  
rt-4.1.13.Final.jar:4.1.13.Final]  
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transpo  
rt-4.1.13.Final.jar:4.1.13.Final]  
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport  
-4.1.13.Final.jar:4.1.13.Final]  
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1334) [netty-transport-4.1.13.Final.jar:4.1.13.Final]  
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]  
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]  
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:926) [netty-transport-4.1.13.Final.jar:4.1.13.Final]  
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:134) [netty-transport-4.1.13.Final.jar:4.1.13.Final]  
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:644) [netty-transport-4.1.13.Final.jar:4.1.13.Final]  
at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:544) [netty-transport-4.1.13.Final.jar:4.1.13.Final]  
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:498) [netty-transport-4.1.13.Final.jar:4.1.13.Final]  
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:458) [netty-transport-4.1.13.Final.jar:4.1.13.Final]  
at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858) [netty-common-4.1.13.Final.jar:4.1.13.Final]  
at java.lang.Thread.run(Thread.java:748) [?:1.8.0\_151]  
Caused by: javax.net.ssl.SSLException: Received fatal alert: bad\_certificate  
at sun.security.ssl.Alerts.getSSLException(Alerts.java:208) ~[?:?]  
at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1666) ~[?:?]  
at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1634) ~[?:?]  
at sun.security.ssl.SSLEngineImpl.recvAlert(SSLEngineImpl.java:1800) ~[?:?]  
at sun.security.ssl.SSLEngineImpl.readRecord(SSLEngineImpl.java:1083) ~[?:?]  
at sun.security.ssl.SSLEngineImpl.readNetRecord(SSLEngineImpl.java:907) ~[?:?]  
at sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:781) ~[?:?]  
at javax.net.ssl.SSLEngine.unwrap(SSLEngine.java:624) ~[?:1.8.0\_151]  
at io.netty.handler.ssl.SslHandler$SslEngineType$3.unwrap(SslHandler.java:255) ~[?:?]  
at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1162) ~[?:?]  
at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1084) ~[?:?]  
at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:489) ~[?:?]  
at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:428) ~[?:?]  
... 15 more

```
	[2018-01-04T06:22:13,630][WARN][o.e.x.s.t.n.SecurityNetty4Transport] [node02] client did not trust this server's certificate, closing connection org.elasticsearch.transport.netty4.NettyTcpChannel@5f423fb7

```

[2018-01-04T06:22:13,829][WARN][o.e.x.s.t.n.SecurityNetty4HttpServerTransport] [node02] caught exception while handling client http traffic, closing connection [id: 0xc7f5d548, L:0.0.0.0/0.0.0.0:9200 ! R:/10.81.101.226:51106]  
io.netty.handler.codec.DecoderException: io.netty.handler.ssl.NotSslRecordException: not an SSL/TLS record: 48454144202f20485454502f312e310d0a436f6e6e656374696f6e3a204b6565702d416c6976650d0a436f6e74656e742d547970653a206170706c69636174696f6e2f6a736f6e0d0a486f73743a207669656c6b30326473792e64736f6e652e3364732e636f6d3a393230300d0a557365722d4167656e743a204d616e7469636f726520302e362e310d0a4163636570742d456e636f64696e673a20677a69702c6465666c6174650d0a0d0a  
at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:459) ~[netty-codec-4.1.13.Final.jar:4.1.13.Final]

\*Looks like i havent generated certs properly . Please help in resolving the same .

Let me know if any other information is required .

---

<div class="post-metadata">

**Author:** ![SJN8](https://avatars.discourse-cdn.com/v4/letter/s/f08c70/32.png) [@SJN8](https://discuss.elastic.co/u/SJN8)\
**Post date:** [January 4, 2018, 12:16pm UTC](https://discuss.elastic.co/t/x-pack-installation-to-elk-stack/113944/5 "2018-01-04T12:16:43Z")

</div>

Recently i have updated shard allocation settings as well ,  
below are the details ,

########################################### Shard allocation #############################################

cluster.routing.allocation.enable: all  
#cluster.routing.allocation.node\_concurrent\_incoming\_recoveries: 2  
#cluster.routing.allocation.node\_concurrent\_outgoing\_recoveries: 2  
cluster.routing.allocation.node\_concurrent\_recoveries: 2  
cluster.routing.allocation.node\_initial\_primaries\_recoveries: 4  
cluster.routing.allocation.same\_shard.host: false

################################ Shard rebalancing #######################

cluster.routing.rebalance.enable: all  
cluster.routing.allocation.allow\_rebalance: always  
cluster.routing.allocation.cluster\_concurrent\_rebalance:2

############################# shard balancing Heuristcs #################

cluster.routing.allocation.balance.shard: 0.45f  
cluster.routing.allocation.balance.index: 0.55f  
cluster.routing.allocation.balance.threshold: 1.0f

######################## Disk Based Shard Allocation ##########################

cluster.routing.allocation.disk.threshold\_enabled: true  
cluster.routing.allocation.disk.watermark.low: 85%  
cluster.routing.allocation.disk.watermark.high: 90%  
cluster.routing.allocation.disk.watermark.flood\_stage: 95%  
cluster.info.update.interval: 50s  
cluster.routing.allocation.disk.include\_relocations: true

but still not working .

elasticsearch[21489]: [2018-01-04T13:10:02,414][WARN][o.e.x.s.t.n.SecurityNetty4Transport] [Node01] client did not trust this server's certificate, closing connection org.elasticsearch.transport.netty4.NettyTcpChannel@121c68c4

please help on the same

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 1, 2018, 12:16pm UTC](https://discuss.elastic.co/t/x-pack-installation-to-elk-stack/113944/6 "2018-02-01T12:16:51Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
