X509 certificate signed by unknown authority - received "bad_certificate"

Hi,

I came across some erros with metricbeat and the elasticsearch-xpack module.

Here is the error i get from metricbeat :

Nov 28 12:03:27 S0CO000ELS04 metricbeat[4636]: 2022-11-28T12:03:27.827+0100#011ERROR#011module/wrapper.go:259#011 Error fetching data for metricset elasticsearch.enrich: error determining if connected Elasticsearch node is master: error making http request: Get "https://S0CO000ELS04.entity.com:9200/_nodes/_local/nodes": x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "Entity Signing CA")

And in the cluster logs :

eAddress=/172.20.50.64:44188}                                                                                                                                                                                      
io.netty.handler.codec.DecoderException: javax.net.ssl.SSLHandshakeException: Received fatal alert: bad_certificate                                                                                                
       at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:477) ~[netty-codec-4.1.66.Final.jar:4.1.66.Final]                                                                      
       at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276) ~[netty-codec-4.1.66.Final.jar:4.1.66.Final]                                                                     
       at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.66.Final.jar:4.1.66.Final]                                                
       at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.66.Final.jar:4.1.66.Final]                                                
       at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.66.Final.jar:4.1.66.Final]                                                  
       at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) [netty-transport-4.1.66.Final.jar:4.1.66.Final]                                                       
       at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.66.Final.jar:4.1.66.Final]                                                
       at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.66.Final.jar:4.1.66.Final]                                                
       at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) [netty-transport-4.1.66.Final.jar:4.1.66.Final]                                                                
       at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) [netty-transport-4.1.66.Final.jar:4.1.66.Final]                                                         
       at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:719) [netty-transport-4.1.66.Final.jar:4.1.66.Final]                                                                             
       at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:620) [netty-transport-4.1.66.Final.jar:4.1.66.Final]                                                                       
       at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:583) [netty-transport-4.1.66.Final.jar:4.1.66.Final]                                                                            
       at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) [netty-transport-4.1.66.Final.jar:4.1.66.Final]
       at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) [netty-common-4.1.66.Final.jar:4.1.66.Final]
       at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) [netty-common-4.1.66.Final.jar:4.1.66.Final]
       at java.lang.Thread.run(Thread.java:1589) [?:?]
Caused by: javax.net.ssl.SSLHandshakeException: Received fatal alert: bad_certificate
       at sun.security.ssl.Alert.createSSLException(Alert.java:130) ~[?:?]
       at sun.security.ssl.Alert.createSSLException(Alert.java:117) ~[?:?]
       at sun.security.ssl.TransportContext.fatal(TransportContext.java:358) ~[?:?]
       at sun.security.ssl.Alert$AlertConsumer.consume(Alert.java:286) ~[?:?]
       at sun.security.ssl.TransportContext.dispatch(TransportContext.java:204) ~[?:?]
       at sun.security.ssl.SSLTransport.decode(SSLTransport.java:172) ~[?:?]
       at sun.security.ssl.SSLEngineImpl.decode(SSLEngineImpl.java:736) ~[?:?]
       at sun.security.ssl.SSLEngineImpl.readRecord(SSLEngineImpl.java:691) ~[?:?]
       at sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:506) ~[?:?]
       at sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:482) ~[?:?]
       at javax.net.ssl.SSLEngine.unwrap(SSLEngine.java:679) ~[?:?]
       at io.netty.handler.ssl.SslHandler$SslEngineType$3.unwrap(SslHandler.java:298) ~[netty-handler-4.1.66.Final.jar:4.1.66.Final]
       at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1344) ~[netty-handler-4.1.66.Final.jar:4.1.66.Final]
       at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1237) ~[netty-handler-4.1.66.Final.jar:4.1.66.Final]
       at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1286) ~[netty-handler-4.1.66.Final.jar:4.1.66.Final]
       at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:507) ~[netty-codec-4.1.66.Final.jar:4.1.66.Final]
       at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:446) ~[netty-codec-4.1.66.Final.jar:4.1.66.Final]

Looks like a bad certificate or a CA issue so i tried to verify that the cert is issued by :

root@S0CO000ELS04: openssl verify -verbose -CAfile /etc/elasticsearch/certs/entity-ca.crt /etc/elasticsearch/certs/S0CO000ELS04.entity.com.crt 
/etc/elasticsearch/certs/S0CO000ELS04.entity.com.crt: OK

Here is my metricbeat config :

output.elasticsearch:
  hosts: ["array"]
  # Protocol - either `http` (default) or `https`.
  protocol: "https"

  # Authentication credentials - either API key or username/password.
  #api_key: "id:api_key"
  username: "metricbeat"
  password: ""
  ssl.verification: none # Even with this SSL errors still persits
output.elasticsearch.ssl.certificate_authorities: ["/etc/elasticsearch/certs/Entity-ca.crt"]

The module :

- module: elasticsearch
  xpack.enabled: true
  period: 10s
  hosts: ["https://S0CO000ELS04.entity.com:9200"]
  username: "metricbeat"
  password: ""
  ssl.enabled: true
  ssl.certificate_authorithies: ["/etc/elasticsearch/certs/Entity-ca.crt"]
  ssl.verification: none

There is something i'm missing about this error and the configuration ?

I dont know why it's unknown certificate authority for the same certificate used by my cluster

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.