# Filebeat not harvesting all files in a folder

**URL:** <https://discuss.elastic.co/t/filebeat-not-harvesting-all-files-in-a-folder/153293>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [October 21, 2018, 5:27pm UTC](https://discuss.elastic.co/t/filebeat-not-harvesting-all-files-in-a-folder/153293 "2018-10-21T17:27:34Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Srinivasa\_Prasad](https://avatars.discourse-cdn.com/v4/letter/s/bcef8e/32.png) [@Srinivasa\_Prasad](https://discuss.elastic.co/u/Srinivasa_Prasad)\
**Post date:** [October 21, 2018, 5:27pm UTC](https://discuss.elastic.co/t/filebeat-not-harvesting-all-files-in-a-folder/153293/1 "2018-10-21T17:27:34Z")

</div>

I have a log folder with 744 files, but filebeat has read only 155 files. Is there a setting that controls max number of files to be read.

I am using docker-compose and mapped bind mount volume of local log directory to filebeat docker container. And the container path is configured in the filebeat.yml file.

Please suggest.

---

<div class="post-metadata">

**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\
**Post date:** [October 22, 2018, 2:42pm UTC](https://discuss.elastic.co/t/filebeat-not-harvesting-all-files-in-a-folder/153293/2 "2018-10-22T14:42:32Z")

</div>

Can you share your filebeat configuration?

Any errors in filebeat logs?

Is there an `ulimit` on maximum number of files filebeat or any process is allowed to open?

There is a `harvester_limit` setting, but I don't think it's set by default.

---

<div class="post-metadata">

**Author:** ![rugenl](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rugenl/32/12887_2.png) [@rugenl](https://discuss.elastic.co/u/rugenl)\
**Post date:** [October 22, 2018, 3:19pm UTC](https://discuss.elastic.co/t/filebeat-not-harvesting-all-files-in-a-folder/153293/3 "2018-10-22T15:19:10Z")

</div>

Other things; permissions, file name pattern, age of file (ignore\_older).

---

<div class="post-metadata">

**Author:** ![Srinivasa\_Prasad](https://avatars.discourse-cdn.com/v4/letter/s/bcef8e/32.png) [@Srinivasa\_Prasad](https://discuss.elastic.co/u/Srinivasa_Prasad)\
**Post date:** [October 30, 2018, 11:57am UTC](https://discuss.elastic.co/t/filebeat-not-harvesting-all-files-in-a-folder/153293/4 "2018-10-30T11:57:19Z")

</div>

Thanks. I found that files already read are not accessed again.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 27, 2018, 11:57am UTC](https://discuss.elastic.co/t/filebeat-not-harvesting-all-files-in-a-folder/153293/5 "2018-11-27T11:57:30Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.