But I've tried this, and I used Filebeat fingerprint processor, filebeat fails to update the document in elasticsearch if it already have the same id, while logstash does update the document successfully.
I've read some comments in the discussion forms here about using logstash for such purpose as filebeat is not designed for this, but that's not what the elastic guide says!
What I understand, if two documents have the same "_id" then these are considered duplicates, the last document inserted should overwrite the previous one.
I'm running things in my lab for now, the configuration is very simple, reading a .csv file, you can't get it wrong, filebeat is inserting docs successfully, no issues, it's just when I insert a new line with same "myid", but slightly different info in other fields, it fails to update.
This is my CSV file where filebeat is reading data from, these two events should have updated each other. I tried to push each line by itself, to garantee the order filebeat reads and index the data:
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.