# Filebeat stops harvesting - file didn't change **URL:** \ **Category:** Beats\ **Tags:** filebeat\ **Created:** [May 6, 2016, 11:02pm UTC](https://discuss.elastic.co/t/filebeat-stops-harvesting-file-didnt-change/49446 "2016-05-06T23:02:36Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![knightsg](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/knightsg/32/4648_2.png) [@knightsg](https://discuss.elastic.co/u/knightsg)\ **Post date:** [May 6, 2016, 11:02pm UTC](https://discuss.elastic.co/t/filebeat-stops-harvesting-file-didnt-change/49446/1 "2016-05-06T23:02:36Z")
Today I found out about, and tried out filebeat for sending logs to Elasticsearch via Logstash, which I already had set up. It seemed to work fine for a bit, until I noticed that FB stops sending logs after a period of activity. Here's my setup: - Ubuntu 14.04 - Filebeat 1.2.2 - Logstash 2.2 - Elasticsearch 1.5.2 Using tcpdump I determined that the instance running Filebeat had completely stopped sending data to the logstash instances, so I then turned on debug logging in Filebeat and discovered that it thinks the logs it's watching are not being updated, even though they are. I primarily noticed this with my apache logs which are constantly receiving traffic. However, Filebeat reports: 2016-05-06T22:55:25Z DBG scan path /var/log/apache2/access.log 2016-05-06T22:55:25Z DBG Check file for harvesting: /var/log/apache2/access.log 2016-05-06T22:55:25Z DBG Update existing file for harvesting: /var/log/apache2/access.log 2016-05-06T22:55:25Z DBG Not harvesting, file didn't change: /var/log/apache2/access.log I notice that apache constantly updates the modification time on access.log as it writes to it, and I assume the same happens with the other logs I'm monitoring. It appears that Filebeat is getting confused by this behaviour. I did some googling and found that this seems to have been a problem for others, but so far I haven't found a solution. I tried setting 'spool\_size: 1' for each prospector, but it did nothing. I also set registry\_file: .filebeat for each prospector, this is the output of it: {"/var/log/apache2/access.log":{"source":"/var/log/apache2/access.log","FileStateOS":{"inode":154945,"device":51713}},"/var/log/apache2/error.log":{"source":"/var/log/apache2/error.log","FileStateOS":{"inode":154946,"device":51713}},"/var/log/php/php.log":{"source":"/var/log/php/php.log","FileStateOS":{"inode":271523,"device":51713}},"/var/log/syslog":{"source":"/var/log/syslog","FileStateOS":{"inode":154666,"device":51713}}} I've tried watching this file to see if it changes, but it doesn't. If I restart filebeat it works for a short while and then starts showing that "Not harvesting, file didn't change" error. Does anyone have a solution for this? Thanks, Guy ---
**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\ **Post date:** [May 9, 2016, 7:25am UTC](https://discuss.elastic.co/t/filebeat-stops-harvesting-file-didnt-change/49446/2 "2016-05-09T07:25:52Z")
Are the logs by coincidence on a mounted volume? Filebeat uses the modification date decide if a file should be continued to read, so it is good that apache keeps updating it. What are your values for ignore\_older and close\_older? ---
**Author:** ![Todd\_Ruch](https://avatars.discourse-cdn.com/v4/letter/t/73ab20/32.png) [@Todd\_Ruch](https://discuss.elastic.co/u/Todd_Ruch)\ **Post date:** [July 14, 2016, 1:59am UTC](https://discuss.elastic.co/t/filebeat-stops-harvesting-file-didnt-change/49446/3 "2016-07-14T01:59:53Z")
This looks very similar to my issue. > [@Filebeat still scanning files, but not finding new log entries](https://discuss.elastic.co/t/filebeat-still-scanning-files-but-not-finding-new-log-entries/55424): > > Hello! I'm using filebeat-1.2.3-1.x86\_64 on a CentOS 6.5 host. The problem I'm having is that after I start filebeat through the init process, it runs for a short period (what seems like the initial batch) and then anthough it is still running (I see it running due to it being set to log level debug), it stops finding new log entries. I don't seem to have the same problem when running it in debug in the foreground. filebeat -c /etc/filebeat/filebeat.yml -e -v -d '\*' filebeat: prospectors: … ---
**Author:** ![leoleoleo](https://avatars.discourse-cdn.com/v4/letter/l/aca169/32.png) [@leoleoleo](https://discuss.elastic.co/u/leoleoleo)\ **Post date:** [July 14, 2016, 7:54am UTC](https://discuss.elastic.co/t/filebeat-stops-harvesting-file-didnt-change/49446/4 "2016-07-14T07:54:03Z")
I have exactly the same issue. Filbeat correctly send events when starting but then stop the logs because `Not harvesting, file didn't change` even if the log file has change This seems to occurs on busy file which are update every seconds. Normal files like `syslog` seems to be correctly handled by filebeat filebeat: `1.2.3` logstash: `2.3.4-1` ---
**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\ **Post date:** [July 17, 2016, 8:03pm UTC](https://discuss.elastic.co/t/filebeat-stops-harvesting-file-didnt-change/49446/5 "2016-07-17T20:03:48Z")
@leoleoleo any chance to test if you encounter still the same issue with the most recent nightly build? [https://beats-nightlies.s3.amazonaws.com/index.html?prefix=filebeat/](https://beats-nightlies.s3.amazonaws.com/index.html?prefix=filebeat/) ---
**Author:** ![knightsg](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/knightsg/32/4648_2.png) [@knightsg](https://discuss.elastic.co/u/knightsg)\ **Post date:** [July 20, 2016, 5:50pm UTC](https://discuss.elastic.co/t/filebeat-stops-harvesting-file-didnt-change/49446/6 "2016-07-20T17:50:00Z")
I meant to update this ages ago, but this actually started working for me a bit after I posted this issue. I have no idea what changed, but since then it's been fine. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 5, 2017, 9:50pm UTC](https://discuss.elastic.co/t/filebeat-stops-harvesting-file-didnt-change/49446/7 "2017-07-05T21:50:53Z")