# Logstash rereading the files even after since\_db has that file entry which was already read **URL:** \ **Category:** Logstash\ **Created:** [November 18, 2024, 7:25am UTC](https://discuss.elastic.co/t/logstash-rereading-the-files-even-after-since-db-has-that-file-entry-which-was-already-read/370675 "2024-11-18T07:25:39Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![venkatkumar229](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/venkatkumar229/32/104663_2.png) [@venkatkumar229](https://discuss.elastic.co/u/venkatkumar229)\ **Post date:** [November 18, 2024, 7:25am UTC](https://discuss.elastic.co/t/logstash-rereading-the-files-even-after-since-db-has-that-file-entry-which-was-already-read/370675/1 "2024-11-18T07:25:39Z")
Hi Team, I am trying to read the files from a particular path using logstash file input plugin and i am using **file\_completed\_path and since\_db** path to track the list of files which are processed. It was working fine and logstash is continuously updating both the **file\_completed\_path, since\_db** with the files and reading only new files. But suddenly from today it started reading the old files and creating a duplicate entries for the same files in **file\_completed\_path, since\_db** path. There are no modifications to **file\_completed\_path, since\_db** paths and using them from starting. Could you please suggest what may be the route cause for this issue and why logstash is reading the files again even they are already read and updated in the since\_db path. Thanks in advance. ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [November 18, 2024, 12:50pm UTC](https://discuss.elastic.co/t/logstash-rereading-the-files-even-after-since-db-has-that-file-entry-which-was-already-read/370675/2 "2024-11-18T12:50:22Z")
Can you show us an example of a duplicated entry? ---
**Author:** ![venkatkumar229](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/venkatkumar229/32/104663_2.png) [@venkatkumar229](https://discuss.elastic.co/u/venkatkumar229)\ **Post date:** [November 18, 2024, 12:59pm UTC](https://discuss.elastic.co/t/logstash-rereading-the-files-even-after-since-db-has-that-file-entry-which-was-already-read/370675/3 "2024-11-18T12:59:25Z")
Hi @Badger, Please find the below sample entries in the file\_completed\_log\_path folder for my pipeline. The same file has read twice. One read happened on Nov 12 and another one Today(Nov 18). /var/log/Elastic/sas\_data/sas\_elastic\_orders\_dump\_20241112.csv /var/log/Elastic/sas\_data/sas\_elastic\_orders\_dump\_20241112.csv ---
**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\ **Post date:** [November 18, 2024, 1:06pm UTC](https://discuss.elastic.co/t/logstash-rereading-the-files-even-after-since-db-has-that-file-entry-which-was-already-read/370675/4 "2024-11-18T13:06:43Z")
I think you need to share the duplicate entries in the sincedb file, showing the inode as well. Is `/var/log/Elastic/sas_data ` a NFS share? What is creating those files? Also, share your file input as well. ---
**Author:** ![venkatkumar229](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/venkatkumar229/32/104663_2.png) [@venkatkumar229](https://discuss.elastic.co/u/venkatkumar229)\ **Post date:** [November 18, 2024, 1:23pm UTC](https://discuss.elastic.co/t/logstash-rereading-the-files-even-after-since-db-has-that-file-entry-which-was-already-read/370675/5 "2024-11-18T13:23:45Z")
Hi @leandrojmp, Please find the below entries from since\_db file. 6450506898 0 41 1432982 1731840644.927283 /data/apac/nfs/elk/sas\_elastic\_orders\_dump\_20241112.csv 6450506898 0 40 1432982 1731922241.158001 /data/apac/nfs/elk/sas\_elastic\_orders\_dump\_20241112.csv Yes, we are getting these files from NFS share which is mounted on our logstash server. File input plugin: ```auto input { file { id => "sas-orders-input" path => "/data/apac/nfs/elk/sas_elastic_orders_dump_*.csv" mode => "read" file_completed_action => "log" file_completed_log_path => "/var/log/Elastic/sas_data" start_position => "beginning" sincedb_path => "/var/log/Elastic/db_logstash/sas_data" } } ``` Thank you in Advance. ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [November 18, 2024, 1:50pm UTC](https://discuss.elastic.co/t/logstash-rereading-the-files-even-after-since-db-has-that-file-entry-which-was-already-read/370675/6 "2024-11-18T13:50:41Z")
The device number has changed from 40 to 41, which means logstash thinks it is a different disk. That can happen with NFS mounts when they are re-mounted (e.g. after a reboot). There's nothing logstash can do about it. You may be able to fix it for the client/server (e.g. [here](https://docs.redhat.com/en/documentation/red_hat_enterprise_linux/6/html/logical_volume_manager_administration/persistent_numbers)) but that will be OS specific. ---
**Author:** ![venkatkumar229](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/venkatkumar229/32/104663_2.png) [@venkatkumar229](https://discuss.elastic.co/u/venkatkumar229)\ **Post date:** [November 18, 2024, 3:13pm UTC](https://discuss.elastic.co/t/logstash-rereading-the-files-even-after-since-db-has-that-file-entry-which-was-already-read/370675/7 "2024-11-18T15:13:53Z")
Hi @Badger , Thank you for the update. Now the NFS team is planning to migrate the FS to Azure NFS after they did the changes again will our logstash reprocess all the files. Is there any way to handle this situation without rerunning the old files. We are not creating any lvcreate to create volumes we are just mounting it in our server using fstab. XXXX:/data/ /nas\_data nfs defaults 0 0 Our OS Details: RHEL 7.9(mapio) ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [November 18, 2024, 5:48pm UTC](https://discuss.elastic.co/t/logstash-rereading-the-files-even-after-since-db-has-that-file-entry-which-was-already-read/370675/8 "2024-11-18T17:48:36Z")
I don't think you can affect the device numbers using fstab.