# How to send only new line data in filebeats

**URL:** <https://discuss.elastic.co/t/how-to-send-only-new-line-data-in-filebeats/126418>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [April 2, 2018, 10:51am UTC](https://discuss.elastic.co/t/how-to-send-only-new-line-data-in-filebeats/126418 "2018-04-02T10:51:46Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Kuysuperall](https://avatars.discourse-cdn.com/v4/letter/k/e480ec/32.png) [@Kuysuperall](https://discuss.elastic.co/u/Kuysuperall)\
**Post date:** [April 2, 2018, 10:51am UTC](https://discuss.elastic.co/t/how-to-send-only-new-line-data-in-filebeats/126418/1 "2018-04-02T10:51:46Z")

</div>

hi i try to config filebeat like a if in CSV or log file it have new line record it must to send only new line record not all off them data  
this is my config

```auto
filebeat.prospectors:
- type: log
  enabled: true
  paths:
      - D:\Filebeat\datafrombeat.csv
    #- c:\programdata\elasticsearch\logs\*
   ignore_older: 12h
   close_older: 1m
   close_inactive: 1m
   clean_removed: true
   close_removed: true
   scan_frequency: 10s
   registry_file: registry

```

thank you

---

<div class="post-metadata">

**Author:** ![andrewkroh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrewkroh/32/3784_2.png) [@andrewkroh](https://discuss.elastic.co/u/andrewkroh)\
**Post date:** [April 3, 2018, 1:38pm UTC](https://discuss.elastic.co/t/how-to-send-only-new-line-data-in-filebeats/126418/2 "2018-04-03T13:38:21Z")

</div>

> [@Kuysuperall](#):
>
> hi i try to config filebeat like a if in CSV or log file it have new line record it must to send only new line record not all off them data

Are you saying that Filebeat is resending the whole file rather than just the newly added log lines?

If so, that would mean that Filebeat is not "remembering" that it's reading from the same file. So either it's not the same file (the whole file is being re-written each time it's updated and the file has a new inode) OR Filebeat is cleaning its persisted state for the inode at some point.

How is the CSV file being updated? Filebeat expects log files to be appended to.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 1, 2018, 1:38pm UTC](https://discuss.elastic.co/t/how-to-send-only-new-line-data-in-filebeats/126418/3 "2018-05-01T13:38:23Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.