Metricbeat kafka module kerberos authentication

Hi
Am trying to enable metricbeat kafka module in my kafka cluster. My kafka cluster uses kerberos auth to access all the topics. Here is my current config:

# Module: kafka
# Docs: https://www.elastic.co/guide/en/beats/metricbeat/7.10/metricbeat-module-kafka.html

# Kafka metrics collected using the Kafka protocol
- module: kafka
  #metricsets:
  #  - partition
  #  - consumergroup
  period: 10s
  hosts: ["localhost:9092"]

  client_id: metricbeat
  #retries: 3
  #backoff: 250ms

  # List of Topics to query metadata for. If empty, all topics will be queried.
  #topics: []

  # Optional SSL. By default is off.
  # List of root certificates for HTTPS server verifications
  #ssl.certificate_authorities: ["/etc/pki/root/ca.pem"]

  # Certificate for SSL client authentication
  #ssl.certificate: "/etc/pki/client/cert.pem"

  # Client Certificate Key
  #ssl.key: "/etc/pki/client/cert.key"

  # SASL authentication
  #username: ""
  #password: ""
  kerberos.auth_type: keytab
  kerberos.config_path: /etc/krb5.conf
  kerberos.keytab: /etc/metricbeat/certs/ex_elk.keytab
  kerberos.service_name: kafka
  kerberos.realm: NONPROD.REALM
# Metrics collected from a Kafka broker using Jolokia
- module: kafka
  metricsets:
    - broker
    - producer
    - consumer
  period: 10s
  hosts: ["localhost:8778"]

Am wondering if this is supported using kerberos authentication for kafka module since I don't see it in the document guides.

Hello! I think this is added by https://github.com/elastic/beats/pull/16871. Is this what you are looking for? Thanks!

Hi thank you for replying

Yes I've done that if you can see my config above.
But am getting this error:

2021-02-24T14:56:59.342+0800    INFO    module/wrapper.go:259   Error fetching data for metricset kafka.consumergroup: error in connect: getting cluster client for advertised broker with address 10.244.17.22:9092: kafka: client has run out of available brokers to talk to (Is your cluster reachable?)
2021-02-24T14:57:09.414+0800    INFO    module/wrapper.go:259   Error fetching data for metricset kafka.consumergroup: error in connect: getting cluster client for advertised broker with address 10.244.17.22:9092: kafka: client has run out of available brokers to talk to (Is your cluster reachable?)

I've test the keytab file and am able to get all the topics.

@kvch Do you know how to fix this issue? TIA!!

Unfortunately, Kerberos authentication is not yet added to kafka module in Metricbeat.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.