I have a filebeat.yml that works perfectly sending csv files to logstash, and then logstash to elastic.
If I add a new file to any of the folders filebeats starts it's magic. But if I add data to the end of a existing/read file, filebeats reads the hole file from start including new line. What I'm looking for is that filebeat reads the new added data for existing files. Is this possible?
Here is my filebeat.yml:
filebeat: prospectors: - paths: - "/home/vagrant/files/type_one*.csv" document_type: type_one ignore_older: 1m - paths: - "/home/vagrant/files/type_two*.csv" document_type: type_two ignore_older: 1m - paths: - "/home/vagrant/files/type_three*.csv" document_type: type_three ignore_older: 1m output: logstash: hosts: ["127.0.0.1:5044"]