How to send only new line data in filebeats

[Kuysuperall]

hi i try to config filebeat like a if in CSV or log file it have new line record it must to send only new line record not all off them data
this is my config

filebeat.prospectors:
- type: log
  enabled: true
  paths:
      - D:\Filebeat\datafrombeat.csv
    #- c:\programdata\elasticsearch\logs\*
   ignore_older: 12h
   close_older: 1m
   close_inactive: 1m
   clean_removed: true
   close_removed: true
   scan_frequency: 10s
   registry_file: registry

thank you

[andrewkroh]

@Kuysuperall

hi i try to config filebeat like a if in CSV or log file it have new line record it must to send only new line record not all off them data

Are you saying that Filebeat is resending the whole file rather than just the newly added log lines?

If so, that would mean that Filebeat is not "remembering" that it's reading from the same file. So either it's not the same file (the whole file is being re-written each time it's updated and the file has a new inode) OR Filebeat is cleaning its persisted state for the inode at some point.

How is the CSV file being updated? Filebeat expects log files to be appended to.