# Logstash is restarting when trying to reload certificate
**URL:** https://discuss.elastic.co/t/logstash-is-restarting-when-trying-to-reload-certificate/345100
**Category:** Logstash
**Created:** [October 16, 2023, 12:10pm UTC](https://discuss.elastic.co/t/logstash-is-restarting-when-trying-to-reload-certificate/345100 "2023-10-16T12:10:39Z")
**Posts on this page:** 10
**Page:** 1
### Author:  [@djrshn2346](https://discuss.elastic.co/u/djrshn2346)
#### Post date: [October 16, 2023, 12:10pm UTC](https://discuss.elastic.co/t/logstash-is-restarting-when-trying-to-reload-certificate/345100/1 "2023-10-16T12:10:39Z")
I am trying to reduce the TTL period to 10 mins and then after 8 mins, the certificate should reload but it does not happens, then it fails and restarts the Logstash, this process restart also fails and the complete pod restarts then runs. Why it is not able to reload, and then not able to restart logstash process?
Thanks in advance.
---
### Author:  [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)
#### Post date: [October 16, 2023, 12:26pm UTC](https://discuss.elastic.co/t/logstash-is-restarting-when-trying-to-reload-certificate/345100/2 "2023-10-16T12:26:38Z")
Hello,
Which certiicate?
> [@djrshn2346](#):
>
> then it fails and restarts the Logstash, this process restart also fails and the complete pod restarts then runs.
Please share your logs, it is not possible to know what is happening without the logs.
---
### Author:  [@djrshn2346](https://discuss.elastic.co/u/djrshn2346)
#### Post date: [October 16, 2023, 12:40pm UTC](https://discuss.elastic.co/t/logstash-is-restarting-when-trying-to-reload-certificate/345100/3 "2023-10-16T12:40:22Z")
It is the internal certificate.
```auto
2023-10-02T03:23:54.411913167+02:00 io.netty.handler.codec.DecoderException: javax.net.ssl.SSLHandshakeException: Received fatal alert: bad_certificate
2023-10-02T03:23:54.414756675+02:00 at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:477) ~[netty-all-4.1.65.Final.jar:4.1.65.Final]
2023-10-02T03:23:54.417504825+02:00 at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276) ~[netty-all-4.1.65.Final.jar:4.1.65.Final]
2023-10-02T03:23:54.420340431+02:00 at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-all-4.1.65.Final.jar:4.1.65.Final]
2023-10-02T03:23:54.423383790+02:00 at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-all-4.1.65.Final.jar:4.1.65.Final]
2023-10-02T03:23:54.426008991+02:00 at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-all-4.1.65.Final.jar:4.1.65.Final]
2023-10-02T03:23:54.429181907+02:00 at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) [netty-all-4.1.65.Final.jar:4.1.65.Final]
2023-10-02T03:23:54.432767742+02:00 at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-all-4.1.65.Final.jar:4.1.65.Final]
2023-10-02T03:23:54.436461652+02:00 at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-all-4.1.65.Final.jar:4.1.65.Final]
2023-10-02T03:23:54.440534061+02:00 at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) [netty-all-4.1.65.Final.jar:4.1.65.Final]
2023-10-02T03:23:54.444140699+02:00 at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) [netty-all-4.1.65.Final.jar:4.1.65.Final]
2023-10-02T03:23:54.447798668+02:00 at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:719) [netty-all-4.1.65.Final.jar:4.1.65.Final]
2023-10-02T03:23:54.451207607+02:00 at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:655) [netty-all-4.1.65.Final.jar:4.1.65.Final]
2023-10-02T03:23:54.455150572+02:00 at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:581) [netty-all-4.1.65.Final.jar:4.1.65.Final]
2023-10-02T03:23:54.458290019+02:00 at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) [netty-all-4.1.65.Final.jar:4.1.65.Final]
2023-10-02T03:23:54.461293026+02:00 at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) [netty-all-4.1.65.Final.jar:4.1.65.Final]
2023-10-02T03:23:54.464633120+02:00 at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) [netty-all-4.1.65.Final.jar:4.1.65.Final]
2023-10-02T03:23:54.467576752+02:00 at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [netty-all-4.1.65.Final.jar:4.1.65.Final]
2023-10-02T03:23:54.470472000+02:00 at java.lang.Thread.run(Thread.java:829) [?:?]
2023-10-02T03:23:54.473210104+02:00 Caused by: javax.net.ssl.SSLHandshakeException: Received fatal alert: bad_certificate
2023-10-02T03:23:54.475945917+02:00 at sun.security.ssl.Alert.createSSLException(Alert.java:131) ~[?:?]
2023-10-02T03:23:54.479015120+02:00 at sun.security.ssl.Alert.createSSLException(Alert.java:117) ~[?:?]
2023-10-02T03:23:54.482143176+02:00 at sun.security.ssl.TransportContext.fatal(TransportContext.java:347) ~[?:?]
2023-10-02T03:23:54.485006887+02:00 at sun.security.ssl.Alert$AlertConsumer.consume(Alert.java:293) ~[?:?]
2023-10-02T03:23:54.487733459+02:00 at sun.security.ssl.TransportContext.dispatch(TransportContext.java:186) ~[?:?]
2023-10-02T03:23:54.490434446+02:00 at sun.security.ssl.SSLTransport.decode(SSLTransport.java:172) ~[?:?]
2023-10-02T03:23:54.493404712+02:00 at sun.security.ssl.SSLEngineImpl.decode(SSLEngineImpl.java:681) ~[?:?]
2023-10-02T03:23:54.495967547+02:00 at sun.security.ssl.SSLEngineImpl.readRecord(SSLEngineImpl.java:636) ~[?:?]
2023-10-02T03:23:54.498900917+02:00 at sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:454) ~[?:?]
2023-10-02T03:23:54.501594004+02:00 at sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:433) ~[?:?]
2023-10-02T03:23:54.504536298+02:00 at javax.net.ssl.SSLEngine.unwrap(SSLEngine.java:637) ~[?:?]
2023-10-02T03:23:54.507572022+02:00 at io.netty.handler.ssl.SslHandler$SslEngineType$3.unwrap(SslHandler.java:298) ~[netty-all-4.1.65.Final.jar:4.1.65.Final]
2023-10-02T03:23:54.510362065+02:00 at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1338) ~[netty-all-4.1.65.Final.jar:4.1.65.Final]
2023-10-02T03:23:54.513165468+02:00 at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1234) ~[netty-all-4.1.65.Final.jar:4.1.65.Final]
2023-10-02T03:23:54.516214898+02:00 at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1280) ~[netty-all-4.1.65.Final.jar:4.1.65.Final]
2023-10-02T03:23:54.518965928+02:00 at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:507) ~[netty-all-4.1.65.Final.jar:4.1.65.Final]
2023-10-02T03:23:54.521813272+02:00 at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:446) ~[netty-all-4.1.65.Final.jar:4.1.65.Final]
2023-10-02T03:23:54.529303415+02:00 ... 17 more
```
It shows bad certificate error when tries to reload the new certificate which is being found.
---
### Author:  [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)
#### Post date: [October 16, 2023, 12:45pm UTC](https://discuss.elastic.co/t/logstash-is-restarting-when-trying-to-reload-certificate/345100/4 "2023-10-16T12:45:44Z")
> [@djrshn2346](#):
>
> It is the internal certificate.
Which internal certificate? The certitificate used in Elasticsearch output? The certificate used on an input? Is this the CA or the client certificate?
> [@djrshn2346](#):
>
> `2023-10-02T03:23:54.473210104+02:00 Caused by: javax.net.ssl.SSLHandshakeException: Received fatal alert: bad_certificate`
This error is pretty clear, there is something wrong with the certificate.
Whar certificate is this? It is still not clear as you can use a certificate in many places.
Can you share your logstash configuration?
---
### Author:  [@djrshn2346](https://discuss.elastic.co/u/djrshn2346)
#### Post date: [October 16, 2023, 12:51pm UTC](https://discuss.elastic.co/t/logstash-is-restarting-when-trying-to-reload-certificate/345100/5 "2023-10-16T12:51:16Z")
Input client cert.
---
### Author:  [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)
#### Post date: [October 16, 2023, 12:59pm UTC](https://discuss.elastic.co/t/logstash-is-restarting-when-trying-to-reload-certificate/345100/6 "2023-10-16T12:59:31Z")
As mentioned before, there is something wrong in the certificate you need to double check it.
Also, you didn't share any configuration nor any log showing that Logstash is restarting, can you share the logs showing that logstash is restarting?
---
### Author:  [@djrshn2346](https://discuss.elastic.co/u/djrshn2346)
#### Post date: [October 17, 2023, 5:27am UTC](https://discuss.elastic.co/t/logstash-is-restarting-when-trying-to-reload-certificate/345100/7 "2023-10-17T05:27:15Z")
sorry, cant share the complete logs, but sharing some of the parts.
```auto
2023-10-02T03:24:59.574215917+02:00 {"version": "1.0.0", "timestamp": "2023-10-02T01:24:59.570+00:00", "severity": "info", "service_id": "eric-log-transformer", "metadata": {"proc_id": "supervisor.config_reload.sh", "pod_name": "eric-log-transformer-7d4744b89-9h9sd", "pod_uid": "624a9dbc-5581-4c56-9351-6e9d526f7f72", "container_name": "logtransformer", "node_name": "node-10-63-134-150", "namespace": "footprint"}, "message": "New certificate found /run/secrets/input-cert/srvcert.pem with validity: notBefore=Oct 2 01:23:09 2023 GMT; notAfter=Oct 2 01:33:39 2023 GMT"}
2023-10-02T03:25:18.307072430+02:00 {"version": "1.0.0", "timestamp": "2023-10-02T01:25:18.304+00:00", "severity": "error", "service_id": "eric-log-transformer", "metadata": {"proc_id": "supervisor.monitor_input_certs.sh", "pod_name": "eric-log-transformer-7d4744b89-9h9sd", "pod_uid": "624a9dbc-5581-4c56-9351-6e9d526f7f72", "container_name": "logtransformer", "node_name": "node-10-63-134-150", "namespace": "footprint"}, "message": "Certificate reload failure detected, Logstash input pipeline is using an expired certificate since at least 301 seconds ago. Requesting supervisor to restart Logstash."}
2023-10-02T03:25:18.722045532+02:00 {"version": "1.0.0", "timestamp": "2023-10-02T01:25:18.717+00:00", "severity": "error", "service_id": "eric-log-transformer", "metadata": {"proc_id": "supervisor.init.sh", "pod_name": "eric-log-transformer-7d4744b89-9h9sd", "pod_uid": "624a9dbc-5581-4c56-9351-6e9d526f7f72", "container_name": "logtransformer", "node_name": "node-10-63-134-150", "namespace": "footprint"}, "message": "Restarting Logstash as instructed by certificate monitor."}
2023-10-02T03:25:18.733909392+02:00 {"version": "1.1.0", "timestamp": "2023-10-02T01:25:18.731Z", "severity": "warning", "service_id": "eric-log-transformer", "metadata" : {"namespace": "footprint", "pod_name": "eric-log-transformer-7d4744b89-9h9sd", "container_name": "logtransformer"}, "message": "[logstash.runner] SIGTERM received. Shutting down."}
2023-10-02T03:25:37.626742018+02:00 {"version": "1.1.0", "timestamp": "2023-10-02T01:25:37.618Z", "severity": "error", "service_id": "eric-log-transformer", "metadata" : {"namespace": "footprint", "pod_name": "eric-log-transformer-7d4744b89-9h9sd", "container_name": "logtransformer"}, "message": "[logstash.inputs.tcp] null: closing due:"}
2023-10-02T03:25:37.638001518+02:00 io.netty.handler.codec.DecoderException: javax.net.ssl.SSLHandshakeException: Insufficient buffer remaining for AEAD cipher fragment (2). Needs to be more than tag size (16)
2023-10-02T03:24:11.778609895+02:00 {"version": "1.1.0", "timestamp": "2023-10-02T01:24:11.769Z", "severity": "info", "service_id": "eric-log-transformer", "metadata" : {"namespace": "footprint", "pod_name": "eric-log-transformer-7d4744b89-9h9sd", "container_name": "logtransformer"}, "message": "[org.logstash.beats.BeatsHandler] [local: 0.0.0.0:5044, remote: 192.168.43.30:37894] Handling exception: io.netty.handler.codec.DecoderException: javax.net.ssl.SSLHandshakeException: Received fatal alert: bad_certificate (caused by: javax.net.ssl.SSLHandshakeException: Received fatal alert: bad_certificate)"}
2023-10-02T03:27:19.825490172+02:00 {"version": "1.0.0", "timestamp": "2023-10-02T01:27:19.821+00:00", "severity": "critical", "service_id": "eric-log-transformer", "metadata": {"proc_id": "supervisor.init.sh", "pod_name": "eric-log-transformer-7d4744b89-9h9sd", "pod_uid": "624a9dbc-5581-4c56-9351-6e9d526f7f72", "container_name": "logtransformer", "node_name": "node-10-63-134-150", "namespace": "footprint"}, "message": "Logstash has not shut down since termination was requested 121 seconds ago. Terminating Logstash process with SIGKILL and exiting."}
2023-10-02T03:27:19.839602445+02:00 {"version": "1.0.0", "timestamp": "2023-10-02T01:27:19.831+00:00", "severity": "error", "service_id": "eric-log-transformer", "metadata": {"proc_id": "supervisor.init.sh", "pod_name": "eric-log-transformer-7d4744b89-9h9sd", "pod_uid": "624a9dbc-5581-4c56-9351-6e9d526f7f72", "container_name": "logtransformer", "node_name": "node-10-63-134-150", "namespace": "footprint"}, "message": "Terminating children of logstash wrapper process (pid 66) first using SIGKILL."}
2023-10-02T03:27:19.854817406+02:00 {"version": "1.0.0", "timestamp": "2023-10-02T01:27:19.850+00:00", "severity": "info", "service_id": "eric-log-transformer", "metadata": {"proc_id": "supervisor.init.sh", "pod_name": "eric-log-transformer-7d4744b89-9h9sd", "pod_uid": "624a9dbc-5581-4c56-9351-6e9d526f7f72", "container_name": "logtransformer", "node_name": "node-10-63-134-150", "namespace": "footprint"}, "message": "Logstash was terminated and returned exit code 137."}
```
Apart from this, when I increase the TTL period, it works fine at that time.
---
### Author:  [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)
#### Post date: [October 17, 2023, 12:16pm UTC](https://discuss.elastic.co/t/logstash-is-restarting-when-trying-to-reload-certificate/345100/8 "2023-10-17T12:16:17Z")
> [@djrshn2346](#):
>
> ```auto
> 2023-10-02T03:24:59.574215917+02:00 {"version": "1.0.0", "timestamp": "2023-10-02T01:24:59.570+00:00", "severity": "info", "service_id": "eric-log-transformer", "metadata": {"proc_id": "supervisor.config_reload.sh", "pod_name": "eric-log-transformer-7d4744b89-9h9sd", "pod_uid": "624a9dbc-5581-4c56-9351-6e9d526f7f72", "container_name": "logtransformer", "node_name": "node-10-63-134-150", "namespace": "footprint"}, "message": "New certificate found /run/secrets/input-cert/srvcert.pem with validity: notBefore=Oct 2 01:23:09 2023 GMT; notAfter=Oct 2 01:33:39 2023 GMT"}
> 2023-10-02T03:25:18.307072430+02:00 {"version": "1.0.0", "timestamp": "2023-10-02T01:25:18.304+00:00", "severity": "error", "service_id": "eric-log-transformer", "metadata": {"proc_id": "supervisor.monitor_input_certs.sh", "pod_name": "eric-log-transformer-7d4744b89-9h9sd", "pod_uid": "624a9dbc-5581-4c56-9351-6e9d526f7f72", "container_name": "logtransformer", "node_name": "node-10-63-134-150", "namespace": "footprint"}, "message": "Certificate reload failure detected, Logstash input pipeline is using an expired certificate since at least 301 seconds ago. Requesting supervisor to restart Logstash."}
>
> ```
These logs says that your certificate is expired, it seems that there is some issue with the certificate, as mentione before you need to check and fix it.
> [@djrshn2346](#):
>
> Apart from this, when I increase the TTL period, it works fine at that time.
Not sure what is this TTL configuration you mention, can you provide the configuration and where you are changing it?
---
### Author:  [@djrshn2346](https://discuss.elastic.co/u/djrshn2346)
#### Post date: [October 18, 2023, 10:06am UTC](https://discuss.elastic.co/t/logstash-is-restarting-when-trying-to-reload-certificate/345100/9 "2023-10-18T10:06:18Z")
Thanks buddy, there was an issue with the certificate providing service, it was sending tampered certificate.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [November 15, 2023, 10:06am UTC](https://discuss.elastic.co/t/logstash-is-restarting-when-trying-to-reload-certificate/345100/10 "2023-11-15T10:06:47Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.