Filebeat not harvesting all files in a folder

I have a log folder with 744 files, but filebeat has read only 155 files. Is there a setting that controls max number of files to be read.

I am using docker-compose and mapped bind mount volume of local log directory to filebeat docker container. And the container path is configured in the filebeat.yml file.

Please suggest.

Can you share your filebeat configuration?

Any errors in filebeat logs?

Is there an ulimit on maximum number of files filebeat or any process is allowed to open?

There is a harvester_limit setting, but I don't think it's set by default.

Other things; permissions, file name pattern, age of file (ignore_older).

Thanks. I found that files already read are not accessed again.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.