# Filebeat pulling older logs to logstash

**URL:** <https://discuss.elastic.co/t/filebeat-pulling-older-logs-to-logstash/107526>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [November 14, 2017, 10:22am UTC](https://discuss.elastic.co/t/filebeat-pulling-older-logs-to-logstash/107526 "2017-11-14T10:22:21Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Vasu\_Deva\_Reddy\_Gund](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vasu_deva_reddy_gund/32/24182_2.png) [@Vasu\_Deva\_Reddy\_Gund](https://discuss.elastic.co/u/Vasu_Deva_Reddy_Gund)\
**Post date:** [November 14, 2017, 10:22am UTC](https://discuss.elastic.co/t/filebeat-pulling-older-logs-to-logstash/107526/1 "2017-11-14T10:22:21Z")

</div>

i am using file-beat to pull the logs to log-stash and from there to gray-log, its working fine with .csv files and when i am sending .txt files facing issue

example : if one error logged at 6:00 AM that log get pushed to grey-log and another error logged at 9:00 AM and appended on the same log file its pushing older log also to grey-log is there a way to stop that presently i am using below configuration please suggest if any thing missed

ignore\_older: 5m  
close\_inactive: 5m  
close\_renamed: true  
close\_removed: true  
close\_eof: true  
clean\_inactive: 10m  
clean\_removed: true  
#exclude\_lines: ["^|"]  
#exclude\_lines: ['^(._?|)_$']  
#exclude\_files: ["\_Info.txt$","\_Debug.txt$","MemCache\_Error.txt$","UpdateChatInQMemcache\_Error.txt$"]  
#multiline.pattern: '^%{TIMESTAMP\_ISO8601}'  
multiline.pattern: '^[0-9]{4}-[0-9]{2}-[0-9]{2}'  
multiline.negate: true  
multiline.match: after

---

<div class="post-metadata">

**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\
**Post date:** [November 16, 2017, 2:30am UTC](https://discuss.elastic.co/t/filebeat-pulling-older-logs-to-logstash/107526/2 "2017-11-16T02:30:45Z")

</div>

I think your issue is that you use `clean_inactive: 10m` so each file which is read more then `10m` ago the state will be removed and on update read from scratch again.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 14, 2017, 2:30am UTC](https://discuss.elastic.co/t/filebeat-pulling-older-logs-to-logstash/107526/3 "2017-12-14T02:30:48Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.