# Filebeat repeatedly sending old entries in log file

**URL:** <https://discuss.elastic.co/t/filebeat-repeatedly-sending-old-entries-in-log-file/55796>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [July 18, 2016, 6:57pm UTC](https://discuss.elastic.co/t/filebeat-repeatedly-sending-old-entries-in-log-file/55796 "2016-07-18T18:57:29Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![jmtrachy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jmtrachy/32/10943_2.png) [@jmtrachy](https://discuss.elastic.co/u/jmtrachy)\
**Post date:** [July 18, 2016, 6:57pm UTC](https://discuss.elastic.co/t/filebeat-repeatedly-sending-old-entries-in-log-file/55796/1 "2016-07-18T18:57:29Z")

</div>

Hello - just started using filebeat in a very simple setup and am seeing some odd behavior. My configuration is as follows:

```auto
filebeat:
  prospectors:
    -
      paths:
        - /usr/src/app/*.log
      input_type: log
      exclude_files: [".gz$"]
  registry_file: "/usr/src/app/filebeat/registry"

output:
  logstash:
    hosts: ["myserver.com:10200"]
    index: log

shipper:

logging:
  files:
    path: /var/log/filebeat/

```

The behavior I'm noticing is that every time I manually modify a log file every line in the file is sent to logstash. Meaning if I have a log file with two lines in it that have already been logged, then I add one more entry and save, all three lines are sent. It looks like the registry is properly updated - the offset changes each time I modify the file.

Probably something stupid I'm doing - any suggestions?

Here's the registry file:

```auto
{"/usr/src/app/log.log":{"source":"/usr/src/app/log.log","offset":400,"FileStateOS":{"inode":663,"device":31}},"/usr/src/app/log2.log":{"source":"/usr/src/app/log2.log","offset":180,"FileStateOS":{"inode":618,"device":31}}}

```

---

<div class="post-metadata">

**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\
**Post date:** [July 19, 2016, 9:52am UTC](https://discuss.elastic.co/t/filebeat-repeatedly-sending-old-entries-in-log-file/55796/2 "2016-07-19T09:52:59Z")

</div>

how exactly do you modify the file. Have you tried `$ echo 'new line' >> /usr/src/app/log.log`? filebeat tries to detect if file is new or not based on meta-data. some tools/editors do create a completely new file on save and unlink the old one. This is detected by filebeat as new file forcing it to resend all content.

---

<div class="post-metadata">

**Author:** ![jmtrachy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jmtrachy/32/10943_2.png) [@jmtrachy](https://discuss.elastic.co/u/jmtrachy)\
**Post date:** [July 20, 2016, 2:25am UTC](https://discuss.elastic.co/t/filebeat-repeatedly-sending-old-entries-in-log-file/55796/3 "2016-07-20T02:25:38Z")

</div>

I was using vi to edit the file manually - that's probably the case then. It ended up not being a problem when I started appending to the log files using node.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 8, 2016, 6:58pm UTC](https://discuss.elastic.co/t/filebeat-repeatedly-sending-old-entries-in-log-file/55796/4 "2016-08-08T18:58:35Z")

</div>

This topic was automatically closed after 21 days. New replies are no longer allowed.