# Logstash 6.4 send duplicates when logrotate

**URL:** <https://discuss.elastic.co/t/logstash-6-4-send-duplicates-when-logrotate/159894>\
**Category:** Logstash\
**Created:** [December 7, 2018, 9:40am UTC](https://discuss.elastic.co/t/logstash-6-4-send-duplicates-when-logrotate/159894 "2018-12-07T09:40:25Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![Chumu](https://avatars.discourse-cdn.com/v4/letter/c/87869e/32.png) [@Chumu](https://discuss.elastic.co/u/Chumu)\
**Post date:** [December 7, 2018, 9:40am UTC](https://discuss.elastic.co/t/logstash-6-4-send-duplicates-when-logrotate/159894/1 "2018-12-07T09:40:25Z")

</div>

Hi,

I'm using Logstash 6.4 (OS: CentOS 6.8) to send logs from files to a message queue (a bit alike Kafka).

The files are in below structure, every 15 minutes, "access.log" in "second\_dir" will be moved to sub-directory "archive\_dir", and renamed to "access%Y%m%d%H%M.log".

|--top\_dir  
|----second\_dir  
| access.log  
|------archive\_dir  
| access201812071215.log

my logstash configuration is as below:  
input {  
file {  
path =\> "/top\_dir/\*\*/access\*.log"  
start\_position =\> "beginning"  
sincedb\_path =\> "somewhere.sincedb"  
sincedb\_clean\_after =\> 5  
}  
}  
filter {}  
output {}

As per the document detailed, the sincedb will track the state by inode, and the file will not be reread after rotation. But I got exact two duplicates, and the time interval they arrive my message queue is 15 minutes. It seems that it reread the renamed "accessXXXX.log" when it moved from "access.log".

Could there anyone please help on this, I've tried all my efforts (read the document again and again, google, and so on)..., many appreciate!

---

<div class="post-metadata">

**Author:** ![guyboertje](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/guyboertje/32/31592_2.png) [@guyboertje](https://discuss.elastic.co/u/guyboertje)\
**Post date:** [December 7, 2018, 11:15am UTC](https://discuss.elastic.co/t/logstash-6-4-send-duplicates-when-logrotate/159894/2 "2018-12-07T11:15:34Z")

</div>

Please upgrade the file input to version 4.1.8. This is a known bug, now fixed.

`bin/logstash-plugin install logstash-input-file --version 4.1.8`

---

<div class="post-metadata">

**Author:** ![Chumu](https://avatars.discourse-cdn.com/v4/letter/c/87869e/32.png) [@Chumu](https://discuss.elastic.co/u/Chumu)\
**Post date:** [December 7, 2018, 11:19am UTC](https://discuss.elastic.co/t/logstash-6-4-send-duplicates-when-logrotate/159894/3 "2018-12-07T11:19:11Z")

</div>

Thanks for quick reply, but my server bases in a private network, can't access internet.

Is there any installable package? I can download it and put it the server.

BTW: how to check the file input version no?

---

<div class="post-metadata">

**Author:** ![guyboertje](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/guyboertje/32/31592_2.png) [@guyboertje](https://discuss.elastic.co/u/guyboertje)\
**Post date:** [December 7, 2018, 11:25am UTC](https://discuss.elastic.co/t/logstash-6-4-send-duplicates-when-logrotate/159894/4 "2018-12-07T11:25:41Z")

</div>

`bin/logstash-plugin list`

Read [https://www.elastic.co/guide/en/logstash/6.5/offline-plugins.html](https://www.elastic.co/guide/en/logstash/6.5/offline-plugins.html)

---

<div class="post-metadata">

**Author:** ![Chumu](https://avatars.discourse-cdn.com/v4/letter/c/87869e/32.png) [@Chumu](https://discuss.elastic.co/u/Chumu)\
**Post date:** [December 7, 2018, 11:30am UTC](https://discuss.elastic.co/t/logstash-6-4-send-duplicates-when-logrotate/159894/5 "2018-12-07T11:30:48Z")

</div>

Thank you very much! It helps a lot!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [January 4, 2019, 11:30am UTC](https://discuss.elastic.co/t/logstash-6-4-send-duplicates-when-logrotate/159894/6 "2019-01-04T11:30:51Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.