What method do you recommend to remove duplicates in an index, based on @timestamp and a field? somthing like this:
if @timestamp and interface_name are equal, delete one of the document
What method do you recommend to remove duplicates in an index, based on @timestamp and a field? somthing like this:
if @timestamp and interface_name are equal, delete one of the document
Use a fingerprint filter to hash those two together and set the document_id on the elasticsearch output. A document that has the same hash will overwrite an existing document with that id.
© 2020. All Rights Reserved - Elasticsearch
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant logo are trademarks of the Apache Software Foundation in the United States and/or other countries.