How to setup Kafka output in Filebeat

Hi,

I have the following Filebeat configuration;

output.kafka:
  enabled: true
  hosts: ["192.168.0.1:<PORT>"]
  topic: testtopic

What port should I provide here? I am entirely new to Kafka. I have Kafka installed and I have the following config files under config folder;

  • connect-console-sink.properties
  • connect-console-source.properties
  • connect-distributed.properties
  • connect-file-sink.properties
  • connect-file-source.properties
  • connect-log4j.properties
  • connect-standalone.properties
  • consumer.properties
  • log4j.properties
  • producer.properties
  • server.properties
  • tools-log4j.properties
  • zookeeper.properties

I have seen clientPort=2181 in zookeeper.properties and bootstrap.servers=192.168.0.1:9092 in producer.properties. So, what port should I mention in filebeat and where can I get that from?

Thanks.

In hosts setting you normally configure you bootstrapping brokers. These will provide the cluster meta-data used to find all brokers in the cluster. The kafka brokers normally use port 9092 (search for 'listeners' in server.propoerties).

1 Like

Thank you so much @steffens People like you is the most important part of a discussion forum like this. There may be persons who already know solution, but hesitate to share their knowledge / answer questions, regardless of it's gravity, but the thing which is appreciable is your attitude, which made you respond to questions like these.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.