# Count of files in sincedb and file\_completed\_log\_path

**URL:** <https://discuss.elastic.co/t/count-of-files-in-sincedb-and-file-completed-log-path/315907>\
**Category:** Logstash\
**Created:** [October 5, 2022, 6:44pm UTC](https://discuss.elastic.co/t/count-of-files-in-sincedb-and-file-completed-log-path/315907 "2022-10-05T18:44:46Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![INS](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ins/32/92827_2.png) [@INS](https://discuss.elastic.co/u/INS)\
**Post date:** [October 5, 2022, 6:44pm UTC](https://discuss.elastic.co/t/count-of-files-in-sincedb-and-file-completed-log-path/315907/1 "2022-10-05T18:44:46Z")

</div>

Hi Can You explain why I have a so much difference in count of particular daily file in sincedb and file\_complete\_log\_path. It seems that sincedb does not reflect the actual number of reprocessed files, hence there are no entries of these files in the records of the sincedb file, and every day it dumps excess data into elastica for me. Sometimes files of the same name appear in the directory and it does not filter them properly because of this.  
[docker.elastic.co/logstash/logstash:8.2.1](http://docker.elastic.co/logstash/logstash:8.2.1)

> [elastic\_user@host\_b03 sincedb]$ cat .sincedb | grep file\_20221001 | wc -l  
> 387  
> [elastic\_user@host\_b03 sincedb]$ cat .sincedb | grep file\_2022105 | wc -l  
> 0  
> [elastic\_user@host\_b03 sincedb]$ cat .sincedb | grep file\_20221005 | wc -l  
> 562  
> [elastic\_user@host\_b03 sincedb]$ cat .sincedb | grep file\_20221004 | wc -l  
> 561  
> [elastic\_user@host\_b03 sincedb]$ cat .sincedb | grep file\_20221003 | wc -l  
> 508  
> [elastic\_user@host\_b03 sincedb]$ cat .sincedb | grep file\_20221002 | wc -l  
> 279  
> [elastic\_user@host\_b03 sincedb]$  
> [elastic\_user@host\_b03 sincedb]$ cat .sincedb | grep file\_20221001 | wc -l  
> 387  
> [elastic\_user@host\_b03 sincedb]$ cd ..  
> [elastic\_user@host\_b03 logstash7]$ cat fin\_eir.log | grep file\_20221001 | wc -l  
> 545  
> [elastic\_user@host\_b03 logstash7]$ cat fin\_eir.log | grep file\_20221002 | wc -l  
> 544  
> [elastic\_user@host\_b03 logstash7]$ cat fin\_eir.log | grep file\_20221003 | wc -l  
> 542  
> [elastic\_user@host\_b03 logstash7]$ cat fin\_eir.log | grep file\_20221004 | wc -l  
> 545  
> [elastic\_user@host\_b03 logstash7]$ cat fin\_eir.log | grep file\_20221005 | wc -l  
> 545

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [October 5, 2022, 6:50pm UTC](https://discuss.elastic.co/t/count-of-files-in-sincedb-and-file-completed-log-path/315907/2 "2022-10-05T18:50:37Z")

</div>

Could be inode reuse. See [this](https://discuss.elastic.co/t/logstash7-6-data-loss-when-input-file-logrotation-happens/239719/2) thread. There is no good solution.

---

<div class="post-metadata">

**Author:** ![INS](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ins/32/92827_2.png) [@INS](https://discuss.elastic.co/u/INS)\
**Post date:** [October 5, 2022, 6:51pm UTC](https://discuss.elastic.co/t/count-of-files-in-sincedb-and-file-completed-log-path/315907/3 "2022-10-05T18:51:56Z")

</div>

Or should I add the parameter sincedb\_write\_interval to low number, because from default we have 15 sec.

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [October 5, 2022, 7:07pm UTC](https://discuss.elastic.co/t/count-of-files-in-sincedb-and-file-completed-log-path/315907/4 "2022-10-05T19:07:09Z")

</div>

I do not think that will fix it.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 2, 2022, 7:07pm UTC](https://discuss.elastic.co/t/count-of-files-in-sincedb-and-file-completed-log-path/315907/5 "2022-11-02T19:07:43Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.