Filebeat & Logstash with intermediate CA (cert chain) Issues

Hi all,

I have an intermediate CA sign the certs for the filebeat client. I have entered the chain file in logstash configuration for the filebeat input stanza in the certificates-authorities entry.

input {
beats {
port => 5044
ssl => true
ssl_certificate_authorities => ["/usr/local/share/ca-certificates/MySubCA-chain.pem"]
ssl_certificate => "/etc/ssl/certs/logger.abc.co.za.co.za.pem"
ssl_key => "/etc/ssl/private/logger.abc.co.za.key"
ssl_verify_mode => "force_peer"
}
}

On the client I have set up the client certificates and the root CA certificate in the filebeat certificates-authorities.

output.logstash:
hosts: ["logger.abc.co.za:5044"]
ssl.certificate_authorities: ["/usr/local/share/ca-certificates/MySubCA-chain.pem"]
ssl.certificate: "/etc/ssl/certs/client.abc.co.za.pem"
ssl.key: "/etc/ssl/private/client.abc.co.za.key"

I get on the logstash server

"][org.logstash.beats.BeatsHandler] Exception: javax.net.ssl.SSLHandshakeException: General OpenSslEngine problem"

=> on the filebeat client.

"ERR Connecting error publishing events (retrying): remote error: tls: internal error"

Running

"openssl s_client -CAfile /usr/local/share/ca-certificates/MySubCA-chain.pem -servername logger.abc.co.za -connect logger.abc.co.za:5044 -cert /etc/ssl/certs/client.abc.co.za.pem -key /etc/ssl/private/client.abc.co.za..key"

I get

"verify return:1
139717980804760:error:14094438:SSL routines:ssl3_read_bytes:tlsv1 alert internal error:s3_pkt.c:1487:SSL alert number 80
139717980804760:error:140790E5:SSL routines:ssl23_write:ssl handshake failure:s23_lib.c:177:"

Running it without the client certs it works.

"openssl s_client -CAfile /usr/local/share/ca-certificates/MySubCA-chain.pem -servername logger.abc.co.za -connect logger.abc.co.za:5044"

Any ideas whats wrong? According to goolge alert 80 is

internal_error
An internal error unrelated to the peer or the correctness of the

   protocol makes it impossible to continue (such as a memory
   allocation failure). This message is always fatal.

Can you turn on the debug log on Logstash and retry the same scenario and add the output to this issue?

You can do that with this command:

bin/logstash --log.level debug

I am currently assuming that the certificates are correctly signed by the authority and the the CA contains the full chain to correctly validating the certificate.

Thanks for the response. I am confident that the certs are signed properly but with everything in IT I don't rule out the possibility that there is a problem. THe cert and chain work fine with the apache server.

I must be doing something wrong because when I add "--log.level debug" to the systemd service file I get the following error below: (SSL is enabled and the serv ice starts up without the log.level debug option.

] Starting pipeline {"id"=>"main", "pipeline.workers"=>2, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>5, "pipeline.max_inflight"=>250}
[2017-10-31T07:39:19,232][ERROR][logstash.pipeline ] Exception in pipelineworker, the pipeline stopped processing new events, please check your filter configuration and restart Logstash. {"exception"=>"undefined method to_hash' for []:Array", "backtrace"=>["(eval):957:in filter_func'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:398:in filter_batch'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:379:in worker_loop'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:342:in start_workers'"]} [2017-10-31T07:39:19,240][ERROR][logstash.pipeline ] Exception in pipelineworker, the pipeline stopped processing new events, please check your filter configuration and restart Logstash. {"exception"=>"undefined method to_hash' for :Array", "backtrace"=>["(eval):957:in filter_func'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:398:in filter_batch'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:379:in worker_loop'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:342:in start_workers'"]}
[2017-10-31T07:39:19,280][FATAL][logstash.runner ] An unexpected error occurred! {:error=>#<NoMethodError: undefined method to_hash' for []:Array>, :backtrace=>["(eval):957:in filter_func'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:398:in filter_batch'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:379:in worker_loop'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:342:in `start_workers'"]}

Hello,
I just encountered the same issue. In my case, it was caused by the fact that the certificate on Filebeat client had
X509v3 Extended Key Usage: TLS Web Server Authentication
Which means that the certificate is to be used with server, not client.

I've changed the Extended Key Usage to
X509v3 Extended Key Usage: TLS Web Client Authentication
and everything is working flawlessly now!

Let me know if this resolves your issue and have a nice day :slight_smile:

Sorry for the delayed response. I had to refactor our scripts and work out some details with out CA manager to get everything set up for clientAuth extended key usage settings etc but it appears to be working now. thanks for the assistance. Probably a good idea to handle the expectation gracefully in the code?

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.