Logstash does not accept SSL connections from beats

[Ruslan_Hafizov]

I was trying to set up SSL/TLS between beats and input beats in logstash. To create the certificates, I used the following commands:

### creating logstash.p12
elasticsearch-certutil crl --ca /usr/share/elasticsearch/config/certs/elastic-stack-ca.p 12 -in /usr/share/elasticsearch/config/certs/instance.yaml
### logstash.crt
openssl pkcs12 -in logstash.p12 -clcerts -nokeys -out logstash.crt -password pass:yourpassword
### logstash.key
openssl pkcs12 -in logstash.p12 -nocerts -out logstash.key -notes -password pass:yourpassword
### logstash.pkcs8.key
openssl pkcs 8 -topk8 -in logstash.key -out logstash.pkcs 8.key -v2 aes-256-cbc -2gr hmacWithSHA256
### client.crt
openssl pkcs12 -in cert.p12 -clcerts -nokeys -out client.crt -password pass:yourpassword

The following settings were used as input data for generating the p12 storage:

instances:
  - name: "logstash"
    dns:
    - "logstash"
    - "logstashx"
    ip:
    - "10.01.10.10"

I used the following settings for input:

input {
beats {
    port => 5044
    ssl_client_authentication => "required"
    ssl_enabled => true
    ssl_certificate_authorities => ["/usr/share/logstash/config/certs/ca.crt"]
    ssl_certificate => "/usr/share/logstash/config/certs/client.crt"
    ssl_key => "/usr/share/logstash/config/certs/logstash.pkcs8.key"
    ssl_key_passphrase => "***"
  }
}

In output beats, I used different approaches, creating a separate p12 storage in the same way as for logstash, from which I then took out the key and certificate for beats. I also used the logstash certificate itself, but I got SSL errors. The last example of what I tried using the settings:

output.logstash:
  hosts: ["logstash:5044"]
  ssl:
    enabled: true
    certificate_authorities: ["/etc/pki/root/ca.crt"]
    ssl.certificate: "/logstash/certs/client.crt"
    ssl.key: "/logstash/certs/logstash.pkcs8.key"
    ssl_key_passphrase: "***"

I get the following error :

[2025-05-02T19:02:02,741][WARN ][io.netty.channel.DefaultChannelPipeline][main][5036d1d5e0ca7bd9c07af2050b1d7965127518bd086130be02b9f8fd80ece896] An exceptionCaught() event was fired, and it reached at the tail of the pipeline. It usually means the last handler in the pipeline did not handle the exception.
io.netty.handler.codec.DecoderException: javax.net.ssl.SSLHandshakeException: Received fatal alert: bad_certificate
        at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:500) ~[netty-codec-4.1.109.Final.jar:4.1.109.Final]
        at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:290) ~[netty-codec-4.1.109.Final.jar:4.1.109.Final]

Tell me what I'm doing wrong. Thanks

[stephenb (Stephen Brown)]

Hi @Ruslan_Hafizov Welcome to the community.

@Ruslan_Hafizov

pkcs8.key

Edited Corrected!! @Rios is correct! You need the pks8 format on the logstash beats input side

Beats input plugin | Elastic Documentation

But not on the beats output side

[Rios]

Only input beats in LS need ssl_key in the PKCS8 format. It's also on github. Other use the PEM format, LS output and FB output.

[Ruslan_Hafizov]

Thank you all for your answers. This is actually my mistake, but I've also tried this option before. In this case, I was getting a different error :

[2025-05-04T17:17:02,007][WARN ][io.netty.channel.DefaultChannelPipeline][main][b69a2eb3cac144215f5159fc01021d41845460d18e73f609de0fb868bbf430b5] An exceptionCaught() event was fired, and it reached at the tail of the pipeline. It usually means the last handler in the pipeline did not handle the exception.
io.netty.handler.codec.DecoderException: javax.net.ssl.SSLHandshakeException: Empty client certificate chain
        at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:500) ~[netty-codec-4.1.109.Final.jar:4.1.109.Final]

I also used other commands to generate certificates.

openssl genrsa -out beats.key 2048

openssl req -new -key beats.key -out beats.csr -subj "/CN=beats-client"

create beats.ext:

---
authorityKeyIdentifier=keyid,issuer
basicConstraints=CA:FALSE
keyUsage = digitalSignature, nonRepudiation
extendedKeyUsage = clientAuth
---

openssl x509 -req -in beats.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out beats.crt -days 365 -sha256 -extfile beats.ext

#ca.crt and ca.key were extracted from elasticsearch-stack-ca.p12

[stephenb (Stephen Brown)]

You should use the elastic cert util and only use the openssl tool as directed to make the pks8 key.

Here are the comands I ran

$ ./bin/elasticsearch-certutil ca --pem

$ unzip elastic-stack-ca.zip

$ ./bin/elasticsearch-certutil cert --name logstash --ca-cert ./ca/ca.crt --ca-key ./ca/ca.key --dns ip-10-0-0-7 --ip 10.0.0.7 --pem

$ unzip certificate-bundle.zip

$ openssl pkcs8 -inform PEM -in ./logstash/logstash.key -topk8 -nocrypt -outform PEM -out ./logstash/logstash.pkcs8.key

$ ls logstash/
logstash.crt  logstash.key  logstash.pkcs8.key

$ sudo -i 

# cd /etc/logstash/
# mkdir certs
# cp /home/ubuntu/elasticsearch-8.11.1/logstash/* certs/.
# chmod 664 certs/logstash.pkcs8.key
# cp /home/ubuntu/elasticsearch-8.11.1/ca/ca.crt certs/.

[Ruslan_Hafizov]

@stephenb

openssl pkcs8 -inform PEM -in ./logstash/logstash.key -topk8 -nocrypt -outform PEM -out ./logstash/logstash.pkcs8.key

I tried to do what was mentioned above. It didn't help, the error remained the same:

[2025-05-05T07:44:01,838][WARN ][io.netty.channel.DefaultChannelPipeline][main][e22721fd705d0d15e154244309c305d07a4b9acdc0df80c0c3baac2539120e57] An exceptionCaught() event was fired, and it reached at the tail of the pipeline. It usually means the last handler in the pipeline did not handle the exception.
io.netty.handler.codec.DecoderException: javax.net.ssl.SSLHandshakeException: Empty client certificate chain

After that, I re-created the certificates for beats with the following command:

elasticsearch-certutil cert --name beat --ca-cert /usr/share/elasticsearch/config/certs/ca.crt --ca-key /usr/share/elasticsearch/config/certs/ca.key --dns logstash --ip 10.10.10.10 --pem 

Accordingly, I made changes to the configuration and restarted the service.

output.logstash:

  hosts: ["logstash:5044"]
  ssl:
    enabled: true # I also removed and returned this option.
    certificate_authorities: ["/etc/pki/root/ca.crt"]
    ssl.certificate: "/etc/pki/beats/beat.crt"
    ssl.key: "/etc/pki/beats/beat.key"

But the problem remains

[grumo35]

Hey !

Same error message ?

Can we try to debug the SSL part with openssl s_client ?

openssl s_client -connect logstash:5044 -CAfile </path/ca.crt> -cert </path/client.crt> -key </path/client.key>

Please share the output, be aware of potential sensitive information before sharing

[khan1 (khan G)]

I'm trying to set up SSL/TLS between Beats and Logstash using certificates I generated with elasticsearch-certutil and openssl. The Logstash input config expects client authentication, and I provided the necessary certs and keys, including logstash.pkcs8.key and client.crt.

My Logstash input config looks like this:

ruby

CopyEdit

input {
  beats {
    port => 5044
    ssl_enabled => true
    ssl_client_authentication => "required"
    ssl_certificate_authorities => ["/usr/share/logstash/config/certs/ca.crt"]
    ssl_certificate => "/usr/share/logstash/config/certs/client.crt"
    ssl_key => "/usr/share/logstash/config/certs/logstash.pkcs8.key"
    ssl_key_passphrase => 
  }
}

My Beats output config:

yaml

CopyEdit

output.logstash:
  hosts: ["logstash:5044"]
  ssl:
    enabled: true
    certificate_authorities: ["/etc/pki/root/ca.crt"]
    certificate: "/logstash/certs/client.crt"
    key: "/logstash/certs/logstash.pkcs8.key"
    key_passphrase: 

But I keep getting this error:

makefile

CopyEdit
SSLHandshakeException: Received fatal alert: bad_certificate

What's likely wrong?

This error usually means:

The certificate sent by Beats is either not valid, not trusted by Logstash, or
It doesn't match the private key, or
The CA that issued the cert is missing or wrong in ssl_certificate_authorities.

Suggestions:

1. Use a separate certificate for Beats:
   Logstash and Beats should each have their own client certificate signed by the same CA. Don't reuse logstash.pkcs8.key for Beats.
2. Check the CA:
   Make sure the ca.crt used by both Logstash and Beats is the same one that issued the client certs.
3. Verify matching cert/key pair:
   Use this to check:

bash

CopyEdit

openssl x509 -noout -modulus -in client.crt | openssl md5
openssl rsa -noout -modulus -in client.key | openssl md5

They should output the same hash.
4. Set correct permissions on all cert/key files and avoid formatting issues (e.g., no spaces in filenames like pkcs 8.key → rename it to pkcs8.key).

[Ruslan_Hafizov]

I get the following:

CONNECTED(00000003)
Can't use SSL_get_servername
depth=1 CN = Elastic Certificate Tool Autogenerated CA
verify return:1
depth=0 CN = logstash
verify return:1
---
Certificate chain
 0 s:CN = logstash
   i:CN = Elastic Certificate Tool Autogenerated CA
   a:PKEY: rsaEncryption, 2048 (bit); sigalg: RSA-SHA256
   v:NotBefore: May  4 15:48:43 2025 GMT; NotAfter: May  4 15:48:43 2026 GMT
---
Server certificate
-----BEGIN CERTIFICATE-----
...
-----END CERTIFICATE-----
subject=CN = logstash
issuer=CN = Elastic Certificate Tool Autogenerated CA
---
Acceptable client certificate CA names
CN = Elastic Certificate Tool Autogenerated CA
Requested Signature Algorithms: ECDSA+SHA256:ECDSA+SHA384:ECDSA+SHA512:Ed25519:Ed448:RSA-PSS+SHA256:RSA-PSS+SHA384:RSA-PSS+SHA512:RSA-PSS+SHA256:RSA-PSS+SHA384:RSA-PSS+SHA512:RSA+SHA256:RSA+SHA384:RSA+SHA512:ECDSA+SHA1:RSA+SHA1
Shared Requested Signature Algorithms: ECDSA+SHA256:ECDSA+SHA384:ECDSA+SHA512:Ed25519:Ed448:RSA-PSS+SHA256:RSA-PSS+SHA384:RSA-PSS+SHA512:RSA-PSS+SHA256:RSA-PSS+SHA384:RSA-PSS+SHA512:RSA+SHA256:RSA+SHA384:RSA+SHA512
Peer signing digest: SHA256
Peer signature type: RSA-PSS
Server Temp Key: X25519, 253 bits
---
SSL handshake has read 1538 bytes and written 2386 bytes
Verification: OK
---
New, TLSv1.3, Cipher is TLS_AES_256_GCM_SHA384
Server public key is 2048 bit
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
Early data was not sent
Verify return code: 0 (ok)
---
---
Post-Handshake New Session Ticket arrived:
SSL-Session:
    Protocol  : TLSv1.3
    Cipher    : TLS_AES_256_GCM_SHA384
    Session-ID: 2C2063CB9ED35777058C653424221F3948BECAAFBB30B91B1532355D67A9411D
    Session-ID-ctx:
    Resumption PSK: 344F9F1A8296C9FDF2A7A30606020ABF4FD4DE50BD273F5107A5E017DD08F1E366C0A070059446A68DBF7FA22A2DCA60
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    TLS session ticket lifetime hint: 86400 (seconds)
    TLS session ticket:
...

    Start Time: 1746456779
    Timeout   : 7200 (sec)
    Verify return code: 0 (ok)
    Extended master secret: no
    Max Early Data: 0
---
read R BLOCK

[Ruslan_Hafizov]

1. This is one of the options I've been checking.
2. - 4. I checked, everything is fine

openssl verify -CAfile ca.crt ../beats/beat.crt
../beats/beat.crt: OK

[stephenb (Stephen Brown)]

@khan1

output.logstash:
  hosts: ["logstash:5044"]
  ssl:
    enabled: true
    certificate_authorities: ["/etc/pki/root/ca.crt"]
    certificate: "/logstash/certs/client.crt"
    key: "/logstash/certs/logstash.pkcs8.key". 
   ^^^^^ THIS IS NOT CORRECT USE NORMAL NON PKCS8.KEY
    key_passphrase: 

That should be the normal key on the beats side, yeah confusing as @Rios noted above

As shown here

In my example above that would be

output.logstash:
  hosts: ["logstash:5044"]
  ssl:
    enabled: true
    certificate_authorities: ["/etc/pki/root/ca.crt"]
    certificate: "/logstash/certs/client.crt"
    key: "/logstash/certs/logstash.key"

AND I would do all this testing First without pass phrases etc first then add once you get working... the command I gave above work

[Ruslan_Hafizov]

I thank the participants for their answers. My problem was on the output for beats side. I used an extra ssl prefix.