Monitor Kafka using kafka module from metricbeat

[tahseen_fatima (Tahseen)]

Hi Team,

I want to do kafka monitoring using metricbeat kafka module version 7.17v.
I have followed the document -> kafka-module

Configured the output to elasticsearch in metricbeat.yml file.

Executed the below two commands.

kafka-acls --authorizer-properties zookeeper.connect=localhost:2181 --add --allow-principal User:stats --operation Read --topic '*'
kafka-acls --authorizer-properties zookeeper.connect=localhost:2181 --add --allow-principal User:stats --operation Describe --group '*'

- module: kafka
  #metricsets:
  #  - partition
  #  - consumergroup
  period: 10s
  hosts: ["localhost:9092"]
  username: "*********"
  password: "*******"
  ssl.certificate_authorities: ["/etc/metricbeat/certs/kafka_ca.crt"]

But getting below error.

> 2022-09-13T15:44:58.285+0400    ERROR   module/wrapper.go:259   Error fetching data for metricset kafka.consumergroup: error in connect: getting cluster client for advertised broker with address 1*****:9092: kafka: client has run out of available brokers to talk to (Is your cluster reachable?)
> 2022-09-13T15:44:58.285+0400    ERROR   module/wrapper.go:259   Error fetching data for metricset kafka.partition: error in connect: getting cluster client for advertised broker with address 1*****:9092: kafka: client has run out of available brokers to talk to (Is your cluster reachable?)

Kindly help,
Thanks,
Tahseen

[tahseen_fatima (Tahseen)]

Hi,

I have got the solution, I just commented the " ssl.certificate_authorities: ["/etc/metricbeat/certs/kafka_ca.crt"]"
the line and my code started working.

Thanks,
Tahseen