How does Filebeat crawls log files? Inotify?

Elastic Stack Beats
1

Hi :slightly_smiling:

how does Filebeat crawls log files?

Is Filebeat using inotify? Or is Filebeat polling the logs?

Thanks for your time and best wishes
Johannes

2

Filebeat uses polling. See https://github.com/elastic/filebeat/issues/17 (this issue hasn't been migrated to new the elastic/beats repo as far as I can tell).

3

Hi Andrew,

thanks for your quick response! Filebeat is polling for new files.

How does the Filebeat harvester tails a log file? My goal is to estimate the workload and disk IO.

Thanks for your help,
Johannes

4

The harvester reads the file line by line until it reaches EOF. When EOF reached, it backs off until it checks again if new bytes were added from the previously stored offset. Filebeat keeps the file open until the file is older then ignore_older / close_older.

Backoff behaviour and variables can be configured: https://www.elastic.co/guide/en/beats/filebeat/1.1/configuration-filebeat-options.html#_backoff

@Johannes I hope this helps.