Logstash SSL File does not contain a valid private key with Beats

I am trying to setup filebeat to use SSL, but I keep getting errors on the logstash server.

I keep receiving an error when I have my input on logstash set to Beats.

java.lang.IllegalArgumentException: File does not contain valid private key: /etc/logstash/ssl/logstash-proxy.key

Here is my input on my logstash:

input {
  beats {
    #tcp {
    port => 5044

    #ssl_enable => true
    #ssl_verify => false
    #ssl_key => "/etc/logstash/ssl/logstash-proxy.key"
    #ssl_cert => "/etc/logstash/ssl/logstash-proxy.crt"
    #ssl_extra_chain_certs => "/etc/logstash/ssl/ca.com.crt"

    ssl => true
    ssl_certificate_authorities => ["/etc/logstash/ssl/ca.com.crt"]
    ssl_certificate => "/etc/logstash/ssl/logstash-proxy.crt"
    ssl_key => "/etc/logstash/ssl/logstash-proxy.key"
    ssl_verify_mode => "none"
  }
}

If I run this command while logstash input is set to Beats

curl -k -v --cacert logstash-proxy.crt https://10.1.2.143:5044

I receive this error on the filebeat client:

* About to connect() to 10.1.2.143 port 5044 (#0)
*   Trying 10.1.2.143...
* Connected to 10.1.2.143 (10.1.2.143) port 5044 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
* NSS error -5961 (PR_CONNECT_RESET_ERROR)
* TCP connection reset by peer
* Closing connection 0
curl: (35) TCP connection reset by peer

But if I switch over to a TCP configuration, and I run the same curl I receive this on the filebeat client

* About to connect() to 10.1.2.143 port 5044 (#0)
*   Trying 10.1.2.143...
* Connected to 10.1.2.143 (10.1.2.143) port 5044 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
* skipping SSL peer certificate verification
* SSL connection using TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
* Server certificate:
*       subject: CN=*.client.com,E=devops@XYZ.com,OU=ZYZ DevOps Team,O=XYZ,L=City,ST=NY,C=US
*       start date: Dec 05 18:44:03 2018 GMT
*       expire date: Dec 04 18:44:03 2023 GMT
*       common name: *.client.com
*       issuer: CN=ca.com,E=devops@XYZ.com,OU=XYZ DevOps Team,O=XYZ,L=City,ST=NY,C=US
> GET / HTTP/1.1
> User-Agent: curl/7.29.0
> Host: 10.1.2.143:5044
> Accept: */*

If I leave my input set at Beats and I send something from my beats client I receive this error on the logstash console

[2019-03-20T14:24:04,106][WARN ][org.logstash.beats.Server] Exception caught in channel initializer
java.lang.IllegalArgumentException: File does not contain valid private key: /etc/logstash/ssl/logstash-proxy.key

But if I switch to TCP on my logstash input and I try to send something to it using beats I receive this error:

[2019-03-20T14:22:43,832][ERROR][logstash.inputs.tcp      ] Error in Netty pipeline: io.netty.handler.codec.DecoderException: javax.net.ssl.SSLHandshakeException: error:10000412:SSL routines:OPENSSL_internal:SSLV3_ALERT_BAD_CERTIFICATE

I have these certificates working for NXLog -> logstash -> elasticsearch.
Right now I am attempting to replace NXLog with filebeats, and the final piece, which I thought would be the easiest, was to enable SSL. And I can't get it to work.

Any suggestions?

1 Like

I had to convert the .key to pkcs8 by running:

openssl pkcs8 -in logstash-proxy.key -topk8 -out logstash-proxy-pkcs8.key -nocrypt

I then changed my logstash input to be

input {
  beats {
    port => 5044

    ssl => true
    ssl_certificate_authorities => ["/etc/logstash/ssl/ca.com.crt"]
    ssl_certificate => "/etc/logstash/ssl/logstash-proxy.crt"
    ssl_key => "/etc/logstash/ssl/logstash-proxy-pkcs8.key"
    ssl_verify_mode => "none"
  }
}

When I made this change I had an error of

[2019-03-20T15:55:53,695][INFO ][org.logstash.beats.BeatsHandler] [local: 0.0.0.0:5044, remote: 10.1.2.122:53694] Handling exception: Connection reset by peer

When I ran filebeat in verbose I received this error:

2019-03-20T15:45:40.452-0700    ERROR   pipeline/output.go:100  Failed to connect to backoff(async(tcp://10.1.2.143:5044)): x509: cannot validate certificate for 10.1.2.143 because it doesn't contain any IP SANs

This was a quick fix. I added my test server to the hosts file on the local machine and used the "DNS" name in the output on my filebeat.

Everything is now working. Yay!

12 Likes

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.