Filebeat not harvesting all files in a folder


(Srinivasa Prasad) #1

I have a log folder with 744 files, but filebeat has read only 155 files. Is there a setting that controls max number of files to be read.

I am using docker-compose and mapped bind mount volume of local log directory to filebeat docker container. And the container path is configured in the filebeat.yml file.

Please suggest.


(Steffen Siering) #2

Can you share your filebeat configuration?

Any errors in filebeat logs?

Is there an ulimit on maximum number of files filebeat or any process is allowed to open?

There is a harvester_limit setting, but I don't think it's set by default.


(Len Rugen) #3

Other things; permissions, file name pattern, age of file (ignore_older).


(Srinivasa Prasad) #4

Thanks. I found that files already read are not accessed again.


(system) #5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.