SSL handshake error with logstash 7.8.0 kafka integration

Elastic Stack Logstash
1

I'm having a hard time to make the kafka integration to work with logstash 7.8.0 and 7.8.1.

My setup is relatively simple. I'm using self-signed certificates stored in a jks keystore. The logstash input is configured to connect to kafka without endpoint verification. Everything worked well with logstash 6.8.x and even until logstash 7.7.x but with an upgrade to logstash 7.8.0 I'm getting these SSL handshake errors without further details what's failing.

[INFO ][org.apache.kafka.common.network.Selector][elog-parser] [Producer clientId=producer-1] Failed authentication with kafka/172.21.0.3 (SSL handshake failed)
[ERROR][org.apache.kafka.clients.NetworkClient][elog-parser] [Producer clientId=producer-1] Connection to node -1 (kafka/172.21.0.3:9095) failed authentication due to: SSL handshake failed
[WARN ][org.apache.kafka.clients.NetworkClient][elog-parser] [Producer clientId=producer-1] Bootstrap broker kafka:9095 (id: -1 rack: null) disconnected`

Here's the config:

 input {
   kafka {
     bootstrap_servers => "kafka:9095"
     topics => "data-in"
     codec => "json"
     auto_offset_reset => "latest"
     group_id => "logstash"
     security_protocol => "SSL"
     ssl_key_password => "secret"
     ssl_keystore_location => "/usr/share/logstash/ssl/logstash.keystore.jks"
     ssl_keystore_password => "secret"
     ssl_keystore_type => "JKS"
     ssl_truststore_location => "/usr/share/logstash/ssl/kafka.truststore.jks"
     ssl_truststore_password => "secret"
     ssl_truststore_type => "JKS"
     ssl_endpoint_identification_algorithm => ""
   }
}

I even tried to set the log level to debug for the kafka input plugin in log4j2.properties but it doesn't give me more information about what's failing.
Tested with Kafka 5.2.2 and Kafka 5.4.1.

2

In 7.8 the kafka client was updated to 2.4.1. Not sure if that is related.

Can you increase the log level in kafka and see if anything gets logged over there?

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.