# Maybe I missed something: FIlebeat prospectors seems to randomly stop harvesting 'some' files
**URL:** https://discuss.elastic.co/t/maybe-i-missed-something-filebeat-prospectors-seems-to-randomly-stop-harvesting-some-files/37499
**Category:** Beats
**Tags:** filebeat
**Created:** [December 17, 2015, 7:32pm UTC](https://discuss.elastic.co/t/maybe-i-missed-something-filebeat-prospectors-seems-to-randomly-stop-harvesting-some-files/37499 "2015-12-17T19:32:03Z")
**Posts on this page:** 8
**Page:** 1
### Author:  [@Matthew\_Prinvale](https://discuss.elastic.co/u/Matthew_Prinvale)
#### Post date: [December 17, 2015, 7:32pm UTC](https://discuss.elastic.co/t/maybe-i-missed-something-filebeat-prospectors-seems-to-randomly-stop-harvesting-some-files/37499/1 "2015-12-17T19:32:03Z")
This is the 2nd time this has happened now. I have a prospector setup to read /var/log/myapp/\*.log and it brings up about 5 or 6 harvesters as intended. Things run smooth for a while but what seems to be randomly it just stops harvesting one of the files. The file is continually being written to and rotates the same time as all the others. The other prospectors are running just fine and if I restart filebeat it picks the missing file backup and starts shipping again. I don't see any errors in the mybeat logs either.
filebeat.yml
```
filebeat:
registry_file: /var/lib/filebeat/registry
config_dir: /etc/filebeat
output:
logstash:
enabled: true
hosts: ["redacted.com:5044"]
```
prospector#1:
```
filebeat:
prospectors:
-
paths:
- /var/log/myapp/*.log
input_type: log
ignore_older: 10m
document_type: core
```
Here's some log files that might help. Notice the timestamp. After this no info for that particular log is generated (starting harvester, stopping, etc)
mybeat.log:
```
2015-12-17T07:17:55Z ERR File reading error. Stopping harvester. Error: Stop harvesting as file is older then ignore_older: /var/log/myapp/mylog.log; Last change was: 10m5.015760004s
2015-12-17T07:53:45Z ERR File reading error. Stopping harvester. Error: Stop harvesting as file is older then ignore_older: /var/log/myapp/mylog.log; Last change was: 10m5.016134511s
```
/var/log/myapp/mylog.log: Again looking at the timestamps it should be picking this up. I redacted everything except the timestamp and loglevel just to show that the file is being written to.
```
2015-12-17 07:48:57.731427 [INFO]
2015-12-17 07:56:17.676947 [INFO]
2015-12-17 07:56:17.730149 [INFO]
2015-12-17 07:56:17.733369 [INFO]
2015-12-17 07:56:17.734734 [INFO]
2015-12-17 07:56:17.734842 [INFO]
2015-12-17 07:56:17.739996 [INFO]
```
Any ideas on what else I could look at? We are very close to pushing filebeat into production but this 'unknown' is holding it back. Thanks.
---
### Author:  [@steffens](https://discuss.elastic.co/u/steffens)
#### Post date: [December 17, 2015, 9:44pm UTC](https://discuss.elastic.co/t/maybe-i-missed-something-filebeat-prospectors-seems-to-randomly-stop-harvesting-some-files/37499/2 "2015-12-17T21:44:20Z")
which filebeat version are you using.
problem is the ignore\_older option. We set it to 24h by default. Don't know all in details, but [this discussion thread](https://discuss.elastic.co/t/filebeat-do-not-see-file-updates-after-a-while/34914/16) (from message onward) seems to be related.
---
### Author:  [@Matthew\_Prinvale](https://discuss.elastic.co/u/Matthew_Prinvale)
#### Post date: [December 17, 2015, 9:58pm UTC](https://discuss.elastic.co/t/maybe-i-missed-something-filebeat-prospectors-seems-to-randomly-stop-harvesting-some-files/37499/3 "2015-12-17T21:58:39Z")
Hi we are currently using 1.0.0. I just updated to 1.0.1 though after the problem appeared. We have explicitly set the ignore\_older option to 10m.
Even if the file is older then 10 min, once it's written to again won't the scan pick that up and start the harvester back up?
---
### Author:  [@steffens](https://discuss.elastic.co/u/steffens)
#### Post date: [December 18, 2015, 4:48pm UTC](https://discuss.elastic.co/t/maybe-i-missed-something-filebeat-prospectors-seems-to-randomly-stop-harvesting-some-files/37499/4 "2015-12-18T16:48:08Z")
From [this comment](https://discuss.elastic.co/t/filebeat-do-not-see-file-updates-after-a-while/34914/23) it seems ignore\_older should be bigger the maximum update period, so files are not closed too soon and still picked up.
Can you try to run filebeat with -e -d 'prospector' to log some debug message (and other logs) to stderr?
After closing a file due to ignore\_older prospector might generate different debug messages for this file:
```
- (always if last now - mod_time > ignore_older) Fetching old state of file to resume
- Resuming harvester on a previously harvested file
- Skipping file ...
- (if now - mod_time < ignore_older and file renamed) File rename was detected
- (if now - mod_time < ignore_older and no file rename)
- (if file is already known) Resuming harvester on a previously harvested file
- (if new file) Launching harvester on new file
```
The timestamps on your log-file match the actual log event timestamp. Filebeat uses the file modification time. If you know a file its name being affected in advance you can try this line of bash to get unique mod-time updates at about every second:
```
$ logfile=...; while true; do sleep 1; ls -l --full-time $logfile; done | uniq
```
---
### Author:  [@Matthew\_Prinvale](https://discuss.elastic.co/u/Matthew_Prinvale)
#### Post date: [December 18, 2015, 7:38pm UTC](https://discuss.elastic.co/t/maybe-i-missed-something-filebeat-prospectors-seems-to-randomly-stop-harvesting-some-files/37499/5 "2015-12-18T19:38:52Z")
QQ regarding ignore\_older and logrotate. If I increase ignore\_older to something like 1hr and the log file in question gets logrotated, will the prospector pick up the new file right away and start shipping or will it have to wait 1hr for the rotated one to close before picking up the new file?
---
### Author:  [@steffens](https://discuss.elastic.co/u/steffens)
#### Post date: [December 21, 2015, 4:41pm UTC](https://discuss.elastic.co/t/maybe-i-missed-something-filebeat-prospectors-seems-to-randomly-stop-harvesting-some-files/37499/6 "2015-12-21T16:41:28Z")
the rotated file will be picked up I think. But still, the old one will be tried to read until ignore\_older. You can try force\_close\_files option to close the fd before hitting ignore\_older at the cost of missing lines not yet read when file rotation happens.
---
### Author:  [@ruflin](https://discuss.elastic.co/u/ruflin)
#### Post date: [December 27, 2015, 8:58am UTC](https://discuss.elastic.co/t/maybe-i-missed-something-filebeat-prospectors-seems-to-randomly-stop-harvesting-some-files/37499/7 "2015-12-27T08:58:15Z")
The new file will be picked up directly, latest after `scan_frequency`.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 5, 2017, 9:57pm UTC](https://discuss.elastic.co/t/maybe-i-missed-something-filebeat-prospectors-seems-to-randomly-stop-harvesting-some-files/37499/8 "2017-07-05T21:57:12Z")