# Filebeat not detecting new entries in a log file

**URL:** <https://discuss.elastic.co/t/filebeat-not-detecting-new-entries-in-a-log-file/68215>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [December 6, 2016, 8:11pm UTC](https://discuss.elastic.co/t/filebeat-not-detecting-new-entries-in-a-log-file/68215 "2016-12-06T20:11:23Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![romspacenut](https://avatars.discourse-cdn.com/v4/letter/r/96bed5/32.png) [@romspacenut](https://discuss.elastic.co/u/romspacenut)\
**Post date:** [December 6, 2016, 8:11pm UTC](https://discuss.elastic.co/t/filebeat-not-detecting-new-entries-in-a-log-file/68215/1 "2016-12-06T20:11:23Z")

</div>

So we have a service that has an hourly log rotate because the volume of log entries. Filebeat for the most part recognizes that a new file has been created and correctly sees the new entries logged to the log file. There are instances at times where Filebeat will not parse a log file because it thinks there are not new changes to the file. Then when the next hour hits and a new log file is created, it sees those new log entries. Any clues on what may be the cause or should I say increase scan\_frequency? We're currently using 1.3.1.

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [December 6, 2016, 8:17pm UTC](https://discuss.elastic.co/t/filebeat-not-detecting-new-entries-in-a-log-file/68215/2 "2016-12-06T20:17:11Z")

</div>

Try upgrading to 5.0?

What does your config look like?

---

<div class="post-metadata">

**Author:** ![romspacenut](https://avatars.discourse-cdn.com/v4/letter/r/96bed5/32.png) [@romspacenut](https://discuss.elastic.co/u/romspacenut)\
**Post date:** [December 6, 2016, 8:33pm UTC](https://discuss.elastic.co/t/filebeat-not-detecting-new-entries-in-a-log-file/68215/3 "2016-12-06T20:33:29Z")

</div>

Going to hold off on upgrading for now unless it's required to resolve the issue. Here's my config:

```
filebeat:
  prospectors:
    -
      paths:
        - /var/log/messages
        - /var/xyz/business/logs/su/tel-*.log
        - /var/xyz/business/logs/suproxy/tel-*.log
      input_type: log
      multiline:
        # The regexp Pattern that has to be matched. The example pattern matches all lines starting with [
        pattern: ^[0-9]{2}:[0-9]{2}:[0-9]{2}.[0-9]{3}
        negate: true
        match: after
        max_lines: 10000
        timeout: 60s
  registry_file: /var/lib/filebeat/registry
```

---

<div class="post-metadata">

**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\
**Post date:** [December 7, 2016, 9:05am UTC](https://discuss.elastic.co/t/filebeat-not-detecting-new-entries-in-a-log-file/68215/4 "2016-12-07T09:05:09Z")

</div>

Can you share your filebeat log file? It could be that you hit an inode reuse issue which you could solve in 5.0 by using clean\_removed.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [January 4, 2017, 9:05am UTC](https://discuss.elastic.co/t/filebeat-not-detecting-new-entries-in-a-log-file/68215/5 "2017-01-04T09:05:33Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.