Is it possible to read only new / changed log files which come after filebeat start?

Elastic Stack Beats
1

Hi,

I am using ELK GA 6.3.0. I am using Filebeat to read log files from multiple servers and publish them to my Kafka cluster. My log folders already has really huge log files which collectively contains trillions of log entries. There is no issues with Filebeat publishing them to my Kafka topic, but I am facing some difficulties elsewhere. I would like to know if it is possible to parse only new files using filebeat? I mean, Filebeat may publish log entries, which comes after Filebeat start. Is this possible?

Thank you.

2

I have found ignore_older option useful.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.