How to setup Kafka output in Filebeat

Elastic Stack Beats
1

Hi,

I have the following Filebeat configuration;

output.kafka:
  enabled: true
  hosts: ["192.168.0.1:<PORT>"]
  topic: testtopic

What port should I provide here? I am entirely new to Kafka. I have Kafka installed and I have the following config files under config folder;

  • connect-console-sink.properties
  • connect-console-source.properties
  • connect-distributed.properties
  • connect-file-sink.properties
  • connect-file-source.properties
  • connect-log4j.properties
  • connect-standalone.properties
  • consumer.properties
  • log4j.properties
  • producer.properties
  • server.properties
  • tools-log4j.properties
  • zookeeper.properties

I have seen clientPort=2181 in zookeeper.properties and bootstrap.servers=192.168.0.1:9092 in producer.properties. So, what port should I mention in filebeat and where can I get that from?

Thanks.

2

In hosts setting you normally configure you bootstrapping brokers. These will provide the cluster meta-data used to find all brokers in the cluster. The kafka brokers normally use port 9092 (search for 'listeners' in server.propoerties).

1 Like
3

Thank you so much @steffens People like you is the most important part of a discussion forum like this. There may be persons who already know solution, but hesitate to share their knowledge / answer questions, regardless of it's gravity, but the thing which is appreciable is your attitude, which made you respond to questions like these.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.