Discuss the Elastic Stack

How can i process the duplication id or custome document_id on filebeat?

Elastic Stack Beats

beanslee July 5, 2017, 6:34am 1

in logstash ,i can set document_id=xxx
but in file beat， i can not find the way....

thanks!!!!

andrewkroh (Andrew Kroh) July 5, 2017, 2:34pm 2

You cannot set a document ID in Filebeat. You would need to use either Logstash or setup an Ingest Node pipeline to do it.

system (system) Closed August 2, 2017, 2:44pm 3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views	Activity
Setting the Document ID Beats filebeat	7	1642	November 30, 2018
Duplication in Filebeat to Elasticsearch data pushing Beats filebeat	5	702	December 28, 2017
Filebeat deduplication fail to update index Beats filebeat	8	1066	June 18, 2020
Filebeat and updating existing docs Beats filebeat	31	2849	February 6, 2023
How to set "_id" value in elasticsearch document as my custom document id Beats beats-development	6	8305	September 14, 2018

© 2020. All Rights Reserved - Elasticsearch

Elasticsearch is a trademark of Elasticsearch BV, registered in the U.S. and in other countries
Trademarks
Terms
Privacy
Brand
Code of Conduct

Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant logo are trademarks of the Apache Software Foundation in the United States and/or other countries.