Filebeat Kafka output message compression

mruthyu · October 25, 2018, 6:13am

Would like to know whether filebeat compresses the log events before sending to Kafka.

jsoriano · October 29, 2018, 9:13pm

Yes, Kafka output uses compression. Default codec is gzip, and it can be changed with the compression setting.

mruthyu · October 31, 2018, 7:40am

Thanks. So whether the gzip format message is stored in Kafka? And when logstash reads from the kafka input whether it will decompress it?

jsoriano · October 31, 2018, 1:11pm

Yes, the message is stored as sent by the producer and decompressed if needed by the consumer, logstash in this case.

system · November 28, 2018, 3:11pm

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.