Kafka output support in Filebeat. Anyone using Filebeat with Kafka version >2.2.0

Elastic Stack Beats
1

Hi, I'm looking into using Filebeat as an ingress into a pipeline which is based on Kafka. I'm currently using an on-prem Kafka cluster which has brokers running Kafka version 2.5.0 but I see that the highest version supported by Filebeat is 2.2.0. I just need a simple tool to stream log data into Kafka in JSON format.

Has anyone tried using Filebeat with Kafka version 2.5.x ?

2

We are only testing with 2.2 and planning to add 2.3 to our testing matrix. Do you mind opening an issue on Github and ask for support of 2.5? https://github.com/elastic/beats/issues/new

3

Hi, I did test it and it seems to work for my use-case which is to grab loglines from some files and publish them into a Kafka topic for later processing. I will open an issue if I crash into some walls with my setup.

4

No, please open an issue regardless if you see problems. We do not guarantee that Kafka 2.5 works with Beats until we add it to our testing infra. :slight_smile:

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.