# How to know if filebeat has read all the logs in the folder

**URL:** <https://discuss.elastic.co/t/how-to-know-if-filebeat-has-read-all-the-logs-in-the-folder/57899>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [August 12, 2016, 8:24am UTC](https://discuss.elastic.co/t/how-to-know-if-filebeat-has-read-all-the-logs-in-the-folder/57899 "2016-08-12T08:24:55Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![sruthib](https://avatars.discourse-cdn.com/v4/letter/s/47e85d/32.png) [@sruthib](https://discuss.elastic.co/u/sruthib)\
**Post date:** [August 12, 2016, 8:24am UTC](https://discuss.elastic.co/t/how-to-know-if-filebeat-has-read-all-the-logs-in-the-folder/57899/1 "2016-08-12T08:24:55Z")

</div>

1. How log does it basically take to read a file of 1 MB size??  
It is taking around 5 mins for me which is a long time. Is there any parameter that i should keep in mind to make the processing time better.

2. And suppose I have files 5 files that filebeat has to read, is there any order in which the files are picked up and read??

3. And how do we know if all the logs files are read?? Is there any way for the user to know that all the logs are processed looking at something?

---

<div class="post-metadata">

**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\
**Post date:** [August 12, 2016, 2:04pm UTC](https://discuss.elastic.co/t/how-to-know-if-filebeat-has-read-all-the-logs-in-the-folder/57899/2 "2016-08-12T14:04:52Z")

</div>

> 1. How log does it basically take to read a file of 1 MB size??  
> It is taking around 5 mins for me which is a long time. Is there any parameter that i should keep in mind to make the processing time better.

These very slow processing times are normally not fault to filebeat, but to output configured. If target system can not process data faster, filebeat will slow down to not loose any data.

> 1. And suppose I have files 5 files that filebeat has to read, is there any order in which the files are picked up and read??

No particular order. filebeat tries to process all files in parallel

> 1. And how do we know if all the logs files are read?? Is there any way for the user to know that all the logs are processed looking at something?

Enable `info` log in filebeat.yml to print some additional logs about files being visited, #lines being send and files being closed.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 2, 2016, 8:25am UTC](https://discuss.elastic.co/t/how-to-know-if-filebeat-has-read-all-the-logs-in-the-folder/57899/3 "2016-09-02T08:25:21Z")

</div>

This topic was automatically closed after 21 days. New replies are no longer allowed.