Client_id variable

Elastic Stack Beats
1

Can we set the client id as a variable in filebeat.yml file? Here we are using filebeat 6.x

Here is the sample config file below:

#----------------------------- Kafka output --------------------------------
output.kafka:

  # initial brokers for reading cluster metadata
  hosts: ["mybroker1:9092","mybroker2:9092","mybroker3:9092","mybroker4:9092","mybroker5:9092","mybroker6:9092","mybroker7:9092","mybroker8:9092","mybroker9:9092"]

  # message topic selection + partitioning
  topic: '%{[topic]}'
  client_id: '%{[ClientID]}'


- type: log

  paths:
    - /var/log/filebeat/*
    - /var/log/metricbeat/*

  json.message_key:
  json.add_error_key: true
  json.keys_under_root: true

  tail_files: true
  fields_under_root: true
  fields:
    service: abc
    topic: mytopic
    ClientID: myclientid

Error shown : "message":"Invalid kafka configuration: kafka: invalid configuration (ClientID is invalid)"}

2

Hello @savvy,

I've looked the kafka output code and this specific setting is not dynamic, we use it to generate the Kafka client when we create the output and its fixed. It's usually set per application.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.