How to know if filebeat has read all the logs in the folder

Elastic Stack Beats
1

  1. How log does it basically take to read a file of 1 MB size??
    It is taking around 5 mins for me which is a long time. Is there any parameter that i should keep in mind to make the processing time better.

  2. And suppose I have files 5 files that filebeat has to read, is there any order in which the files are picked up and read??

  3. And how do we know if all the logs files are read?? Is there any way for the user to know that all the logs are processed looking at something?

2
  1. How log does it basically take to read a file of 1 MB size??
    It is taking around 5 mins for me which is a long time. Is there any parameter that i should keep in mind to make the processing time better.

These very slow processing times are normally not fault to filebeat, but to output configured. If target system can not process data faster, filebeat will slow down to not loose any data.

  1. And suppose I have files 5 files that filebeat has to read, is there any order in which the files are picked up and read??

No particular order. filebeat tries to process all files in parallel

  1. And how do we know if all the logs files are read?? Is there any way for the user to know that all the logs are processed looking at something?

Enable info log in filebeat.yml to print some additional logs about files being visited, #lines being send and files being closed.

3

This topic was automatically closed after 21 days. New replies are no longer allowed.