# Files are not processed with file input unless I edit them
**URL:** https://discuss.elastic.co/t/files-are-not-processed-with-file-input-unless-i-edit-them/43276
**Category:** Logstash
**Created:** [March 2, 2016, 4:52pm UTC](https://discuss.elastic.co/t/files-are-not-processed-with-file-input-unless-i-edit-them/43276 "2016-03-02T16:52:55Z")
**Posts on this page:** 7
**Page:** 1
### Author:  [@syunusic](https://discuss.elastic.co/u/syunusic)
#### Post date: [March 2, 2016, 4:52pm UTC](https://discuss.elastic.co/t/files-are-not-processed-with-file-input-unless-i-edit-them/43276/1 "2016-03-02T16:52:55Z")
With the file input, I understand that sincedb traces if that file has been processed or not. If I have a new file, that never has being processed by logstash (so is not in sincedb).
My input config is like this:
> input
> {
> file { start\_position =\> "beginning" path =\> "/some/folder/\*.log" add\_field =\> { "client" =\> "myclient" "product" =\> "myprosuct" } }
> }
If I move that file to the folder where logstash is "looking" (/some/folder/\*.log), nothing happened, until I edit that file and change anything inside the file, that file is processed.. the same as it would be in sincedb.
How I know that is not in sincedb? because I execute:
> $ ls -i file.log
> 96993940 /some/folder/file.log
Then:
> $ grep 96993940 /var/lib/logstash/.sincedb\*
and nothing...
Any clues whay this happen? I'm editing log files that each has 3GB... so it's painfully slow.
---
### Author:  [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [March 2, 2016, 6:37pm UTC](https://discuss.elastic.co/t/files-are-not-processed-with-file-input-unless-i-edit-them/43276/2 "2016-03-02T18:37:08Z")
Start Logstash with `--verbose` and check your logs for clues.
---
### Author:  [@syunusic](https://discuss.elastic.co/u/syunusic)
#### Post date: [March 2, 2016, 6:51pm UTC](https://discuss.elastic.co/t/files-are-not-processed-with-file-input-unless-i-edit-them/43276/3 "2016-03-02T18:51:43Z")
I'll do it, when it finishes current files that are being processed. What I did, is a super not fancy way. To every file I ran something like this:
> perl -p -i -e 'BEGIN { print "\n" }' file.log
I don't like it, but it works.
Credits: [http://www.cyberciti.biz/faq/bash-prepend-text-lines-to-file/](http://www.cyberciti.biz/faq/bash-prepend-text-lines-to-file/)
---
### Author:  [@syunusic](https://discuss.elastic.co/u/syunusic)
#### Post date: [March 3, 2016, 6:52pm UTC](https://discuss.elastic.co/t/files-are-not-processed-with-file-input-unless-i-edit-them/43276/4 "2016-03-03T18:52:36Z")
I just find out that old files are not parsed, even If logstash never touched before (old meaning \<1 day old or so). I found it because I had 5 files Logstash never touched it (i.e. weren't in .sincedb\_\*), 2 of those files were new (I created 5 minutes before) and only those 2 were processed. The 3 old ones weren't processed.
It seems almost like there is "something" that scan the whole disk at some point, and "tells" logstash those files are old and only have to touch them if something new arrives to them.
Interesting... and frustrating...
---
### Author:  [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [March 3, 2016, 7:36pm UTC](https://discuss.elastic.co/t/files-are-not-processed-with-file-input-unless-i-edit-them/43276/5 "2016-03-03T19:36:23Z")
Ah, this again. I should've caught it. Look into the file input's ignore\_older option.
---
### Author:  [@syunusic](https://discuss.elastic.co/u/syunusic)
#### Post date: [March 3, 2016, 7:58pm UTC](https://discuss.elastic.co/t/files-are-not-processed-with-file-input-unless-i-edit-them/43276/6 "2016-03-03T19:58:41Z")
That was it. My files are one or two months old, so I put ignore\_older =\> 15552000 (six months).
Thank you!
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 5:08am UTC](https://discuss.elastic.co/t/files-are-not-processed-with-file-input-unless-i-edit-them/43276/7 "2017-07-06T05:08:26Z")