Kafka Module Metricbeat on Kerberized cluster - jaas file

Elastic Stack Beats
, ,
1

Hello,

On our kerberized cloudera hadoop cluster I tried to set up metricbeat kafka module to collect metricsets consumergroup and partitions.
I set up the following kafka.yml

- module: kafka
  period: 10s
  metricsets:
    - consumergroup
    - partition
  hosts: ["localhost:9092"]
  enabled: true


  topics: []

If I mention hosts: ["localhost:9092"], I end up with a "connection refused".
If I specify the ip of the broker, I end up with an ERROR " error.message: broker connection failed: EOF".

As an illustration, below is the config we used to (successfully) consume data from kafka in logstash:

input {
  kafka {
    jaas_path => "/etc/logstash/conf.d/kafka_spark_jaas.conf"
    sasl_kerberos_service_name => "kafka"
    security_protocol => "SASL_PLAINTEXT"
    kerberos_config => "/etc/krb5.conf"
    auto_offset_reset => "earliest"
    topics => ["topicName"]
    codec => "json"
    bootstrap_servers => "xxx.xxx.xxx.xxx:9092,yyy.yyy.yyy.yyy:9092,zzz.zzz.zzz.zzz:9092"
  }
}

So my question is : can we set the same properties to collect kafka metrics in metricbeat kafka module (jaas file, sasl_kerberos_service_name)?

2

Please don't post unformatted text as it is really hard to go through. Especially YAML where the
indentation plays a significant role too.

Instead paste the text and format it with </> icon, and check the preview
window to make sure it's properly formatted before posting it. This makes it
more likely that your question will receive a useful answer.

It would be great if you could update your post to solve this.

3

I've done it.
thx for your reply

4

I'm sorry but you still haven't. We're not trying to be pedantic, but the clearer your post is, the better are the chances for someone to step in and help you solve your problem. For example

is not valid yaml.

5

Sorry ikavas
This time it should be better

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.