Cannot connect metricbeat with an SSL secured Kafka

[JadeJaber (Jade)]

Hi,

I have set the following configuration on my Kafka Module

- module: kafka
  metricsets:
    - partition
    - consumergroup
  period: 10s
  hosts: ["centralfeederkafka01.datalakefeeder###.com:9093"]
  ssl.enabled: true
  ssl.certificate_authorities: ["/usr/hadoop/kafka/config/cert2.truststore"]
  ssl.verification_mode: none
  ssl.certificate: "/usr/hadoop/kafka/config/cert2.pem"
  ssl.key: "/usr/hadoop/kafka/config/cert2.key"

And my certificate chain works

kafka@centralfeederkafka03:/usr/hadoop/metricbeat$ openssl s_client -connect centralfeederkafka01.datalakefeeder.###.com:9093 -key /usr/hadoop/kafka/config/cert2.key -cert /usr/hadoop/kafka/config/cert2.pem -CAfile /usr/hadoop/kafka/config/cert2.truststore
    …
    SSL handshake has read 16671 bytes and written 4664 bytes

    Verification: OK

And here is my Kafka listeners:

listeners=SSL://centralfeederkafka01.datalakefeeder.###.com:9093,PLAINTEXT://centralfeederkafka01.datalakefeeder.###.com:9092
advertised.listeners=SSL://centralfeederkafka01.datalakefeeder.###.com:9093

And here is my current error :

INFO    module/wrapper.go:259   Error fetching data for metricset kafka.partition: error in connect: Could not get cluster client for advertised broker with address centralfeederkafka01.datalakefeeder.###.com:9093

Please note that it is perfectly working on port 9092 with PLAINTEXT

[mtojek (Marcin Tojek)]

Did you try to enable the debug mode and check if there are potential errors in logs?

[JadeJaber (Jade)]

@JadeJaber

error in connect: Could not get cluster client for advertised broker with address

I have just fixed my issue. Actually the certificate should NOT be a wildcard one.